United  States  Government  Accountability  Office 

GAO 

Report  to  Congressional  Committees 

March  2016 

DEFENSE 

ACQUISITIONS 

Assessments  of 
Selected  Weapon 
Programs 

GAO-1 6-329SP 


March  2016 


Gm 

Highlights 

Highlights  of  GAO-1 6-329SP,  a  report  to 
congressional  committees 


Why  GAO  Did  This  Study 

This  is  GAO’s  annual  assessment  of 
DOD  major  weapon  system  acquisitions, 
an  area  on  GAO’s  high-risk  list.  DOD  and 
Congress  have  taken  meaningful  steps  to 
improve  the  acquisition  of  major  weapon 
systems,  yet  programs  continue  to 
experience  cost  and  schedule  overruns. 
Further,  GAO  has  emphasized  the  need 
to  sustain  the  implementation  of 
acquisition  reforms  and  for  programs  to 
complete  developmental  testing  before 
beginning  production,  thereby  avoiding 
concurrency  and  its  associated  cost  and 
schedule  growth.  With  the  continuing 
budgetary  pressures,  DOD  cannot  afford 
to  miss  opportunities  to  address 
inefficiencies  in  these  programs  to  free 
up  resources  for  higher  priority  needs. 

The  joint  explanatory  statement  to  the 
DOD  Appropriations  Act,  2009  includes  a 
provision  for  GAO  to  annually  review 
DOD’s  portfolio  of  weapon  systems.  This 
report  includes  observations  on  (1)  the 
cost  and  schedule  performance  of  DOD’s 
2015  portfolio  of  79  major  defense 
acquisition  programs;  (2)  the  knowledge 
attained  at  key  junctures  in  the 
acquisition  process  for  43  programs  that 
were  in  development  or  early  production; 
and  (3)  key  acquisition  reform  initiatives 
and  program  concurrency.  To  develop 
the  observations  in  this  report,  GAO 
analyzed  cost,  schedule,  and  quantity 
data  from  DOD’s  December  2014 
Selected  Acquisition  Reports.  GAO  also 
collected  data  through  two 
questionnaires  to  program  offices  on 
technology,  design,  and  manufacturing 
knowledge;  the  use  of  knowledge-based 
acquisition  practices;  and  the 
implementation  of  acquisition  reforms 
and  initiatives. 

In  commenting  on  a  draft  of  this  report 
DOD  agreed  with  our  findings  and  noted 
that  our  results  appear  to  validate  DOD’s 
focus  on  continuous  improvements. 
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contact  Michael  J.  Sullivan  at  (202)  512-4841 
or  sullivanm@gao.gov 
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What  GAO  Found 

Over  the  past  year,  the  number  of  programs  in  the  Department  of  Defense’s 
(DOD)  portfolio  of  major  defense  acquisitions  increased  from  78  to  79,  while 
DOD’s  total  planned  investment  in  these  programs  decreased  from  $1.45  trillion 
to  $1.44  trillion.  This  estimate  is  in  line  with  a  trend  seen  since  2010  of  decreases 
in  the  portfolio’s  total  acquisition  cost.  The  portfolio’s  cost  growth  since  first  full 
estimates  has  been  substantial,  but  most  of  the  cost  growth  occurred  5  or  more 
years  ago.  The  average  time  to  deliver  initial  capability  to  the  warfighter  also 
increased  by  2.4  months.  This  increase  is  due  in  part  to  the  significant  delays 
experienced  by  a  few  programs.  In  addition,  while  more  programs  in  the  2015 
portfolio  reported  cost  increases  than  decreases  the  net  change  resulted  in  a 
decrease  in  the  portfolio’s  total  cost  over  the  past  year. 


Distribution  of  the  1-year  Change  in  Total  Acquisition  Cost  within  the  2015  Portfolio 


Percentage  cost  change  interval 


| _ |  Amount  of  cost  change  in  each  percentage  interval  over  the  past  year 

Number  of  programs  in  each  percentage  cost  change  interval 
Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 

Most  of  the  43  programs  GAO  assessed  this  year  are  not  yet  fully  following  a 
knowledge-based  acquisition  approach,  as  GAO  recommended.  This  held  true 
for  the  7  programs  that  recently  entered  system  development  as  none  completed 
all  of  our  criteria  for  a  best  practices  approach.  Each  of  the  7  implemented  some 
knowledge  based  practices — such  as  constraining  the  period  for  development — 
but  other  practices — such  as  fully  maturing  technologies  prior  to  system 
development  start  and  completing  systems  engineering  reviews — were  not  fully 
implemented.  As  a  result,  these  programs  will  carry  unwanted  risk  into 
subsequent  phases  of  acquisition  that  could  result  in  cost  growth  or  schedule 
delays. 

Implementation  of  the  reform  initiatives  GAO  analyzed  varies  for  the  43  programs 
assessed  above  as  well  as  the  12  assessed  that  will  become  programs  in  the 
future.  Programs  are  implementing  acquisition  reform  initiatives — such  as  the  use 
of  affordability  constraints  and  “should  cost”  analysis — and  have  realized  $21 
billion  in  savings  as  a  result.  However,  there  has  been  a  decrease  in  the  number 
of  programs  with  acquisition  strategies  that  include  competition.  In  addition,  a 
number  of  programs  are  concurrently  conducting  both  software  and  hardware 
development  during  production,  exposing  programs  to  undue  cost  and  schedule 
risk. 
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Abbreviations 
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Amphibious  Combat  Vehicle 
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AMDR 

Air  and  Missile  Defense  Radar 

B-2  DMS-M 

B-2  Defensive  Management  System  Modernization 
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Ballistic  Missile  Defense  System 
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Common  Infrared  Countermeasures 
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Defense  Acquisition  Management  Information 
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Department  of  Defense 
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F-15  EPAWSS 

F-15  Eagle  Passive/ Active  Warning  and 

Survivability  System 

F-22  Inc  3.2  B 

F-22  Increment  3.2  B  Modernization 

FAB-T 

Family  of  Advanced  Beyond  Line-of-Sight 

Terminals 

IFPC  Inc  2 

Indirect  Fire  Protection  Capability  Increment  2 

ITEP 

Improved  Turbine  Engine  Program 

JAGM 

Joint  Air-to-Ground  Missile 
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Joint  Light  Tactical  Vehicle 
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Joint  Surveillance  Target  Attack  Radar  System 
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Amphibious  Ship  Replacement 
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Major  Defense  Acquisition  Program 

MGUE 
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manufacturing  readiness  level 

NA 
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NGJ  Inc  1 

Next  Generation  Jammer  Increment  1 

OASuW  Inc  1 

Offensive  Anti-Surface  Warfare  Increment  1 

OR 

Ohio-Class  Replacement 

OMB 

Office  of  Management  and  Budget 

P-8A  Inc  3 

P-8A  Poseidon  Increment  3 

PAR 

Presidential  Aircraft  Recapitalization 
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Selected  Acquisition  Report 

SDB  II 

Small  Diameter  Bomb  Increment  II 

Space  Fence  Inc  1 

Space  Fence  Ground-Based  Radar  System 
Increment  1 
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Ship  to  Shore  Connector  Amphibious  Craft 
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Fleet  Replenishment  Oiler 
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TRL 

technology  readiness  level 

Page  v 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


UCLASS 


Unmanned  Carrier-Launched  Airborne  Surveillance 
and  Strike  system 
WSARA  Weapon  Systems  Acquisition  Reform  Act 

WSF  Weather  Satellite  Follow-On 
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U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


Comptroller  General 
of  the  United  States 


441  G  St.  N.W. 
Washington,  DC  20548 


March  31,  2016 
Congressional  Committees 

I  am  pleased  to  present  GAO’s  14th  annual  assessment  of  the 
Department  of  Defense’s  (DOD)  major  defense  acquisition  programs. 

This  report  offers  observations  on  the  performance  of  DOD’s  current  $1.4 
trillion  portfolio  of  79  programs,  the  smallest  portfolio  in  terms  of  total  cost 
in  a  decade.1  These  79  programs  will  require  roughly  a  third  of  all  DOD’s 
development  and  procurement  funding  over  the  next  5  years.  The  level  of 
investment  these  programs  require  makes  it  imperative  that  DOD 
continue  its  oversight  of  the  implementation  of  key  legislative  and  policy 
reforms  it  began  in  2010  as  well  as  the  implementation  of  the  best 
practices  developed  by  GAO.  In  previous  assessments,  we  reported  that 
some  of  these  reforms  are  addressing  problems  in  DOD  acquisitions.  Our 
current  assessment  shows  that  these  acquisition  reforms  and  GAO’s  best 
practices  are  not  uniformly  implemented  across  the  portfolio.  As  a  result, 
some  programs  continue  to  realize  significant  cost  growth  and  delays  in 
delivering  needed  capability.  Critically,  new  programs  started  each  year 
fulfill  only  some  of  the  best  practices  intended  to  achieve  a  level  of 
knowledge  that  would  demonstrate  they  are  capable  of  meeting  their 
performance  requirements  while  meeting  cost  and  schedule 
commitments.  The  highest  point  of  leverage  is  at  the  start  of  a  new 
program.  Decision  makers  must  ensure  that  new  programs  exhibit 
desirable  principles  before  they  are  approved  and  funded. 

Our  current  assessment  shows  that  DOD  is  making  progress  in 
decreasing  the  amount  of  cost  growth  realized  in  the  portfolio.  When 
compared  to  the  78  major  defense  acquisition  programs  in  last  year’s 
portfolio  the  current  estimated  cost  of  the  79  programs  in  the  2015 
portfolio  is  nearly  $15  billion  less  despite  a  net  increase  in  programs. 
When  assessing  just  the  79  programs  in  the  2015  portfolio  costs  have 


^ur  assessment  of  DOD’s  portfolio  does  not  include  the  cost  of  the  Ballistic  Missile 
Defense  System  (BMDS),  which  we  exclude  as  the  program  lacks  an  acquisition  program 
baseline  needed  to  support  our  assessment  of  cost  and  schedule  change.  For  more 
information  on  BMDS,  see  GAO,  Missile  Defense:  Opportunities  Exist  to  Reduce 
Acquisition  Risk  and  Improve  Reporting  on  System  Capabilities,  GAO- 15-345 
(Washington,  D.C.:  May  6,  2015) 
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decreased  by  $2.5  billion  over  the  past  year,  a  net  decrease  largely 
attributable  to  quantity  reductions  in  the  Patriot  Advanced  Capability-3 
Missile  Segment  Enhancement  program.  In  particular,  programs  that 
started  development  after  the  implementation  of  the  Weapon  Systems 
Acquisition  Reform  Act  of  2009  (WSARA)  and  DOD’s  “Better  Buying 
Power”  initiatives  began  in  2010  have  achieved  cost  reductions  or  shown 
less  cost  growth  than  those  that  began  development  before  2010, 
indicating  that  recent  reforms  may  be  having  a  positive  effect.  This  trend 
is  continued  when  looking  at  the  2015  portfolio’s  $469  billion  in  cost 
growth  since  first  full  estimates  as  $373  billion  of  the  growth  occurred  five 
or  more  years  ago.  Additionally,  older  programs  in  the  portfolio  carry  a 
majority  of  this  growth  compared  to  newer  programs. 

However,  our  analysis  also  highlights  areas  that  still  need  attention  such 
as  the  use  of  knowledge-based  best  practices.  For  example,  some 
programs  are  still  carrying  technology  risk  well  into  system  development 
or  are  proceeding  into  production  before  manufacturing  processes  are 
under  control.  Not  following  knowledge-based  best  practices  puts  these 
programs  at  increased  risk  of  unsatisfactory  acquisition  outcomes.  Our 
assessment  does  show  a  similar  number  of  programs  are  implementing 
initiatives  focused  on  reducing  cost  through  affordability  constraints  and 
efforts  to  find  cost  savings.  In  contrast,  we  also  found  less  use  of 
competition  measures  throughout  the  acquisition  life  cycle  and  overlap 
between  developmental  testing  and  production,  which  threaten  programs’ 
abilities  to  meet  their  cost,  schedule,  and  performance  objectives.  In 
addition,  we  have  a  new  observation  on  the  defense  industrial  base 
showing  that  the  companies  developing  and  delivering  the  largest  DOD 
programs  are  performing  well  in  the  market.  We  plan  to  have  more 
analysis  in  this  area  in  our  next  assessment. 
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Weapon  system  acquisitions  remain  an  area  on  GAO’s  high-risk  list.2 
Continued  efforts  by  DOD  to  mitigate  these  risks  through  adherence  to 
best  practices  and  effective  program  management  are  essential. 


Gene  L.  Dodaro 
Comptroller  General 
of  the  United  States 


2GAO,  High-Risk  Series:  An  Update,  GAO-15-290  (Washington,  D.C.:  February  11, 2015). 
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GAO 

441  G  St.  N.W. 
Washington,  DC  20548 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


March  31, 2016 
Congressional  Committees 

The  joint  explanatory  statement  to  the  Department  of  Defense 
Appropriations  Act,  2009,  included  a  provision  for  GAO  to  annually  review 
the  department’s  $1 .4  trillion  portfolio  of  major  weapon  programs.3  This 
report  also  includes  information  related  to  small  business  participation 
pursuant  to  a  mandate  in  a  report  for  the  National  Defense  Authorization 
Act  for  Fiscal  Year  201 3.4  Additionally,  this  report’s  assessment  of  the 
Amphibious  Combat  Vehicle  (ACV)  along  with  GAO’s  October  2015 
report  on  the  ACV  program  constitute  GAO’s  response  to  the  annual 
reporting  requirement  for  2016  required  by  the  National  Defense 
Authorization  Act  for  Fiscal  Year  2014. 5 

This  report  includes  observations  on  (1)  the  cost  and  schedule 
performance  of  DOD’s  2015  portfolio  of  79  major  defense  acquisition 
programs,  (2)  the  knowledge  attained  at  key  junctures  in  the  acquisition 
process  for  55  current  and  future  weapon  programs  in  development  or 


3See  Explanatory  Statement,  154  Cong.  Rec.  H  9427,  9526  (daily  ed.,  Sept.  24,  2008),  to 
the  Department  of  Defense  Appropriations  Act,  2009,  contained  in  Division  C  of  the 
Consolidated  Security,  Disaster  Assistance,  and  Continuing  Appropriations  Act,  2009, 

Pub.  L.  No.  110-329  (2008). 

4H.R.  Rep.  No.  112-479,  at  284  (2012).  The  National  Defense  Authorization  Act  for  Fiscal 
Year  2013,  Pub.  L.  No. 112-239. 

5GAO,  Amphibious  Combat  Vehicle:  Some  Acquisition  Activities  Demonstrate  Best 
Practices;  Attainment  of  Amphibious  Capability  to  be  Determined,  GAO-16-22 
(Washington,  D.C.:  Oct.  28,  2015).  The  National  Defense  Authorization  Act  for  Fiscal  Year 
2014,  Pub.  L.  No.1 13-66,  §251  (2013). 
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early  production,  and  (3)  key  acquisition  reform  initiatives  and  whether 
programs  are  conducting  or  planning  concurrent  testing  and  production.6 


Our  observations  in  this  report  are  based  on  three  sets  of  programs: 

•  We  assessed  79  major  defense  acquisition  programs  in  DOD’s  2015 
portfolio  for  our  analysis  of  cost  and  schedule  performance.  We 
obtained  cost,  schedule,  and  quantity  data  from  DOD’s  December 
2014  Selected  Acquisition  Reports  (SAR)  and  from  the  Defense 
Acquisition  Management  Information  Retrieval  Purview  (DAMIR) 
system.  We  assessed  the  reliability  of  the  data  by  interviewing 
knowledgeable  agency  officials  on  the  steps  they  take  to  ensure  data 
reliability,  and  determined  that  the  data  were  sufficiently  reliable  for 
the  purposes  of  this  report. 

•  We  assessed  43  major  defense  acquisition  programs  currently 
between  the  start  of  development  and  the  early  stages  of  production 
for  knowledge  attained  at  key  junctures  and  their  implementation  of 
acquisition  reforms.  We  obtained  information  on  the  extent  to  which 
the  programs  follow  knowledge-based  practices — established  by  the 
oeuvre  included  in  the  Related  GAO  Products  section  of  this  report — 
for  technology  maturity,  design  stability,  and  production  maturity  using 
two  data-collection  instruments.  One  was  a  questionnaire  on  issues 
such  as  systems  engineering  reviews,  design  drawings, 
manufacturing  planning  and  execution,  and  the  implementation  of 
specific  acquisition  reforms.  The  other  was  a  data-collection 
instrument  used  to  collect  schedule  dates,  the  programs’  critical 
technology  levels,  and  other  information.  We  received  questionnaire 
responses  from  all  43  current  programs  from  August  through  October 
2015. 

•  We  also  assessed  12  future  major  defense  acquisition  programs  not 
yet  in  the  portfolio  in  order  to  gain  additional  insights  into  knowledge 


6Major  defense  acquisition  programs  (MDAP)  are  those  identified  by  DOD  with  a  dollar 
value  for  all  increments  estimated  to  require  eventual  total  expenditure  for  research, 
development,  test,  and  evaluation  of  more  than  $480  million,  or  for  procurement  of  more 
than  $2.79  billion,  in  fiscal  year  2014  constant  dollars.  DOD  has  a  list  of  programs 
designated  as  future  major  defense  acquisition  programs.  These  programs  have  not 
formally  been  designated  as  MDAPs;  however,  DOD  plans  for  these  programs  to  enter 
system  development,  or  bypass  development  and  begin  production,  at  which  point  they 
will  likely  be  designated  as  MDAPs.  We  refer  to  these  programs  as  future  major  defense 
acquisition  programs  throughout  this  report. 
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attained  before  the  start  of  development  and  their  plans  for 
implementation  of  key  acquisition  reform  initiatives.  We  submitted  a 
questionnaire  to  program  offices  to  collect  information  on  issues  such 
as  program  schedule  events,  costs,  and  numerous  acquisition 
reforms,  and  received  responses  from  all  12  future  programs  from 
July  through  August  2015. 

We  present  individual  assessments  of  55  programs — 43  current  and  12 
future  weapon  programs.  These  assessments  include  major  defense 
acquisition  programs  currently  in  development  or  early  production  as  well 
as  future  programs.  Appendix  I  contains  detailed  information  on  our 
scope  and  methodology. 

We  conducted  this  performance  audit  from  June  2015  to  March  2016  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  based 
on  our  audit  objectives. 

Since  2014,  the  number  of  programs  in  DOD’s  portfolio  of  major  defense 
acquisitions  increased  from  78  to  79,  and  DOD’s  total  planned  investment 
in  these  programs  decreased  by  nearly  $15  billion  from  $1.45  trillion  to 
$1.44  trillion.  This  estimate  is  in  line  with  a  trend  seen  since  2010  of 
decreases  in  the  portfolio’s  total  acquisition  cost.  The  programs  in  the 
2015  portfolio  reported  an  increase  in  the  average  schedule  delay  in 
achieving  initial  capability  of  2.4  months  over  the  past  year.  The  total 
amount  of  funding  needed  to  complete  the  2015  portfolio  is  estimated  at 
$603  billion,  which  is  a  decrease  from  the  2014  portfolio  and  is  the  lowest 
amount  in  over  a  decade.  Of  the  $603  billion,  $571  billion  is  planned  for 
procurement  and  $32  billion  is  planned  for  development.  We  also  found 
that  more  programs  reported  gains  in  buying  power  when  compared  to 
programs  in  the  previous  portfolio.  Despite  the  portfolio’s  overall  cost 
decrease,  42  of  79  programs  increased  in  cost  over  the  past  year  and  the 
entire  portfolio  experienced  $469  billion  in  cost  growth  since  programs 
established  their  first  full  estimates,  primarily  due  to  cost  growth  that 
occurred  5  or  more  years  ago. 

Our  analysis  of  DOD’s  2015  portfolio  allows  us  to  make  the  following  11 
observations. 


Observations  on  the 
Cost  and  Schedule 
Performance  of 
DOD’s  2015  Major 
Defense  Acquisition 
Program  Portfolio 
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Cost  and  Schedule  Performance  Observations 

Changes  across  portfolios 

1.  When  compared  to  the  2014  portfolio,  the  cost  of  the  2015  portfolio  decreased  by  nearly  $15 
billion  from  $1 ,455  to  $1 ,440  billion  and  the  number  of  programs  in  the  portfolio  increased  by  one 
to  79. a 

2.  The  total  amount  of  funding  required  for  the  portfolio  has  been  decreasing  since  201 0.b  The 
amount  of  future  funding  needed  to  complete  the  portfolio — $603  billion — is  at  its  lowest  point  in 
over  a  decade.  Of  the  $603  billion  of  future  funding,  $571  billion  is  for  procurement  and  $32 
billion  is  for  development.  The  low  amount  of  future  development  funding  required  is  likely  due  to 
a  number  of  factors,  including  newer  programs  with  less  risk  entering  system  development. 

Changes  in  the  2015  portfolio 

3.  The  cost  estimates  over  the  past  year  for  the  79  programs  in  the  201 5  portfolio  decreased  by 
$2.5  billion  and  the  average  schedule  delay  in  achieving  initial  capability  increased  by  2.4 
months.  When  assessed  against  first  full  estimates,  total  costs  have  increased  by  $469  billion, 
over  48  percent,  most  of  which  occurred  over  5  years  ago.  The  average  delay  in  delivering  initial 
capabilities  has  increased  to  almost  30  months. 

Factors  that  explain  changes  in  the  2015  portfolio 

4.  Older  programs  in  the  portfolio  carry  a  majority  of  the  total  cost  and  cost  growth  since  first  full 
estimates  when  compared  to  newer  programs.  Of  the  79  programs  in  the  2015  portfolio,  40  were 
also  in  the  2005  portfolio  and  represent  80  percent  of  the  portfolio’s  total  acquisition  cost — over 
$1.1  trillion  of  the  $1.4  trillion  total. 

5.  While  the  total  cost  of  the  2015  portfolio  decreased,  42  of  the  79  programs  increased  in  cost 
over  the  past  year.  Cost  estimate  decreases  on  37  programs  resulted  in  the  overall  net  cost 
decrease. 

6.  Thirty-eight  programs  in  the  portfolio  gained  buying  power  during  the  past  year  resulting  in  a  net 
gain  of  $10.7  billion.  Six  programs  in  our  analysis  have  demonstrated  buying  power  gains  or 
losses  on  an  annual  basis  over  the  past  5  years. 

7.  Schedule  delays  over  the  past  year  in  1 1  of  the  79  programs  contributed  to  the  portfolio’s  overall 
delay  of  2.4  months.  The  Airborne  Maritime/Fixed  Station  radio  system  experienced  the  largest 
delay  at  more  than  8  years,  which  disproportionately  affected  the  portfolio. 

8.  Over  the  past  year,  16  programs  reported  development  cost  growth  while  in  production,  a  phase 
of  the  acquisition  cycle  that  should  have  minimal  development  cost  growth.  This  represents 
concurrency,  albeit  for  different  reasons,  and  is  a  contributor  to  cost  growth. 

Other  Observations 

9.  As  measured  against  the  metrics  discussed  with  the  Office  of  Management  and  Budget  (OMB) 
and  DOD,  more  programs  meet  each  metric  for  cost  change  than  our  last  assessment.  Most 
notably,  72  percent  of  programs  meet  the  threshold  for  less  than  10  percent  growth  over  the  past 
5  years.  Seventy-six  percent  meet  the  threshold  for  less  than  2  percent  growth  in  the  past  year. 
Forty-seven  percent  meet  the  threshold  for  less  than  15  percent  cost  growth  since  first  full 
estimates. 

10.  Army  programs'  percentage  of  the  total  cost  of  the  portfolio  is  the  smallest  of  the  services  and 
has  been  decreasing  since  2007.  The  Navy  and  Air  Force  programs’  percentages  have  been 
increasing  since  2007  and  2012  respectively.  Navy  programs  account  for  almost  55  percent  of 
the  2015  portfolio’s  total  estimated  acquisition  cost. 

1 1 .  The  equity  prices  of  contractors  delivering  the  10  costliest  programs  have  performed  well  relative 
to  broad-based  market  indices,  indicating  that  investors  expect  these  firms  to  remain  profitable 
well  into  the  future. 


aAII  dollar  figures  are  in  fiscal  year  2016  constant  dollars,  unless  otherwise  noted. 
bDetails  on  program  costs  used  for  this  analysis  are  provided  in  appendix  I. 
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Changes  across  Portfolios 


1.  When  compared  to  the  2014  portfolio,  the  cost  of  the  2015 

portfolio  decreased  by  nearly  $15  billion — from  $1,455  to  $1,440 
billion — and  the  number  of  programs  in  the  portfolio  increased 

by  one  to  79.  The  decrease  in  cost  is  due  primarily  to  the  five 
programs  that  exited  the  portfolio  and  accounted  for  $39  billion  in 
spending.  Three  of  these  5  programs  completed  their  planned 
development  and  procurement — the  MH-60S  Fleet  Combat  Support 
Helicopter,  the  National  Airspace  System,  and  the  Patriot  Advanced 
Capability  program — and  accounted  for  over  $24  billion  in  cost.  The 
termination  of  the  Patriot/Medium  Extended  Air  Defense  System 
Combined  Aggregate  Program  and  restructuring  of  the  Joint  Land 
Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor  System 
accounted  for  $14.5  billion  of  the  remaining  $39  billion.  The  6 
programs  entering  the  portfolio  have  a  total  estimated  cost  of  $24 
billion.  The  changes  from  the  2014  portfolio  to  the  2015  portfolio  are 
outlined  in  table  1  below. 


Table  1 :  Cost  Changes  of  DOD’s  Major  Defense  Acquisition  Programs  from  2014  to 
2015 

Fiscal  year  2016  dollars  (in  billions) 

2014  portfolio  (78  programs) 

$1,455 

Less  estimated  total  cost  of  the  5  exiting  programs 

-$39 

Plus  estimated  total  cost  of  the  6  entering  programs 

+$24 

Less  net  cost  changes  on  the  73  remaining  programs 

-$1 

2015  portfolio  (79  programs) 

$1,440 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 


Note:  Our  assessment  of  DOD's  2014  and  2015  portfolios  does  not  include  the  cost  of  the  Ballistic 
Missile  Defense  System  (BMDS),  which  we  exclude  as  the  program  lacks  an  acquisition  program 
baseline  needed  to  support  our  assessment  of  cost  and  schedule  change.  Some  numbers  may  not 
sum  due  to  rounding. 


Six  programs  began  system  development  and  entered  the  portfolio, 
the  Combat  Rescue  Helicopter,  Enhanced  Polar  System, 
Intercontinental  Ballistic  Missile  Fuze  Modernization,  Patriot  Advanced 
Capability-3  Missile  Segment  Enhancement,  Space  Fence  Ground- 
Based  Radar  System  Increment  1,  and  the  VH-92A  Presidential 
Helicopter.  The  Patriot  Advanced  Capability-3  Missile  Segment 
Enhancement  is  leveraging  the  resources  and  development 
conducted  by  the  Patriot/Medium  Extended  Air  Defense  System 
Combined  Aggregate  Program’s  Missile  Unit  sub-element.  Two 
programs — the  Combat  Rescue  Helicopter  and  the  VH-92A 
Presidential  Helicopter — come  from  previously  canceled  acquisition 
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programs.  The  remaining  three  programs  were  in  technology 
development  before  their  entry  into  the  portfolio  was  approved. 

2.  The  total  cost  of  the  portfolio  has  been  decreasing  since  2010. 
The  amount  of  future  funding  needed  to  complete  the  portfolio — 
$603  billion — is  at  its  lowest  point  in  over  a  decade.  Of  the  $603 
billion  of  future  funding,  $571  billion  is  for  procurement  and  only 
$32  billion  is  for  development.  The  low  level  of  remaining 
development  funding  is  likely  due  to  many  factors,  including 
newer  programs  with  less  risk  entering  system  development.  The 
cost  and  number  of  programs  in  the  portfolio  has  decreased  over  the 
past  5  years  as  more  programs  have  exited  than  entered,  the 
programs  entering  are  less  costly,  and  ongoing  programs  are  realizing 
cost  savings.  Currently,  the  total  acquisition  cost  of  the  portfolio  is 
estimated  at  $1.4  trillion,  including  $603  billion  for  future  funding. 
Programs  started  in  the  past  5  years  account  for  just  15  percent  of 
that  $603  billion.  Figure  1  shows  the  changes  in  total  cost  and  number 
of  programs  within  DOD’s  portfolio  of  major  weapon  acquisitions  since 
2005. 


Figure  1:  DOD  Portfolio  Cost  and  Size,  2005-2015 


Fiscal  year  2016  dollars  (in  billions) 


Number  of  programs 
100 


Portfolio  year 

|  Total  acquisition  cost 
— — —  Number  of  programs 
Source:  GAO  analysis  of  DOD  data  |  GAO-16-329SP 


Note:  The  2009  portfolio  is  excluded  because  there  were  no  annual  Selected  Acquisition  Reports 
(SAR)  released  for  the  December  2008  submission  date. 
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The  amount  of  development  and  procurement  funding  needed  to 
complete  the  2015  portfolio,  $571  billion  in  procurement  and  $32 
billion  for  development,  has  been  generally  decreasing.7  By  contrast, 
in  2005  future  funding  was  over  $904  billion,  or  58  percent,  of  the 
portfolio’s  total  cost.  Figure  2  shows  the  change  in  the  portfolio’s 
future  funding  needs  and  funding  invested  as  a  share  of  the  portfolio’s 
total  acquisition  cost  since  2005. 


7Other  costs  include  military  construction  and  operations  and  maintenance,  which  account 
for  $13.1  billion  of  the  2015  portfolio’s  total  acquisition  cost. 
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Figure  2:  DOD  Portfolio  Future  Development  and  Procurement  Funding  in  Comparison  to  Invested  Funding  by  Year,  2005- 
2015 


Fiscal  year  2016  dollars  (in  billions) 
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Source:  GAO  analysis  of  DOD  data.  |  GAO-16-329SP 

Note:  The  2009  portfolio  is  excluded  because  there  were  no  annual  Selected  Acquisition  Reports 
released  in  December  2008. 


The  estimated  cost  to  complete  the  research  and  development 
associated  with  the  portfolio  is  at  the  lowest  point  in  a  decade, 
approximately  $32  billion.  Several  factors  may  have  contributed  to  this 
low  level  of  future  development  funds,  such  as 

•  declining  budgets; 
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•  newer  programs  based  on  more  reasonable  requirements  that 
leverage  mature  technologies  or  modify  existing  designs; 

•  a  majority  of  programs  are  in  early  or  full  production — a  phase  in 
the  acquisition  life  cycle  that  should  require  less  development 
funding;  and 

•  several  programs  starting  system  development  over  the  past  two 
years  have  chosen  potentially  less-risky  or  less  complex 
acquisition  approaches,  such  as  acquiring  capability  in  increments 
and  trading  capability  for  affordability. 


3.  Over  the  past  year,  the  total  acquisition  cost  for  the  79  programs 
in  the  2015  portfolio  decreased  by  $2.5  billion  and  the  average 
schedule  delay  in  achieving  initial  capability  increased  by  2.4 
months.  When  assessed  against  first  full  estimates,  total  costs 
have  increased  by  $469  billion,  over  48  percent,  most  of  which 
occurred  over  5  years  ago.  The  average  delay  in  delivering  initial 
capabilities  has  increased  to  almost  30  months.  While  the  first 
observation  discusses  the  change  from  the  prior  portfolio  of  78 
programs  to  the  current  portfolio  of  79  programs,  this  observation 
addresses  the  change  that  occurred  on  the  79  programs  in  the  current 
portfolio.  Table  2  shows  the  change  in  cost  and  schedule  for  the  2015 
portfolio  in  the  past  year. 


Table  2:  Changes  in  DOD’s  2015  Portfolio  of  79  Major  Defense  Acquisition  Programs  over  the  Past  Year 

Fiscal  year  2016  dollars  (in  billions) 

Estimated 
portfolio  cost  in 
2014 

Estimated 
portfolio  cost  in 
2015 

Estimated 
portfolio  change 
since  2014 

Percentage 
change  since 
2014 

Total  estimated  research  and  development  cost 

$285.9 

$289.0 

$3.1 

1.1% 

Total  estimated  procurement  cost 

1,143.5 

1,137.6 

-6.0 

-0.5 

Total  estimated  acquisition  cost3 

1,442.0 

1,439.6 

-2.5 

-0.2 

Average  delay  in  delivering  initial  capabilities  from 
the  first  full  estimate  of  cost  and  schedule 

27  months 

29.5  months 

2.4  months 
additional  delay 

1.3 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 


aln  addition  to  research  and  development  and  procurement  costs,  total  acquisition  cost  includes 
acquisition-related  operation  and  maintenance  and  system-specific  military  construction  costs.  Some 
numbers  may  not  sum  due  to  rounding. 


Changes  in  the  2015 
portfolio 


The  current  portfolio’s  total  acquisition  cost  has  decreased  over  the 
past  year  by  $2.5  billion,  due  primarily  to  a  $6.0  billion  decrease  in 
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procurement  costs  which  offset  a  $3.1  billion  increase  in  development 
costs.  The  Patriot  Advanced  Capability-3  Missile  Segment 
Enhancement  program  accounts  for  $2  billion  of  the  $2.5  billion 
decrease  due  to  a  quantity  reduction  of  28.5  percent. 

When  measuring  the  current  portfolio’s  schedule  performance  over 
the  past  year,  we  found  that  the  delay  in  delivery  of  initial  operating 
capability  grew  almost  two  and  a  half  months  over  the  past  year  and 
now  stands  at  29.5  months  from  initial  estimates.8 

When  assessed  against  first  full  estimates,  the  total  cost  of  the  2015 
portfolio  increased  by  nearly  $469  billion,  or  48  percent,  most  of  which 
occurred  more  than  5  years  ago.9  These  cost  estimates  and  percent 
growth  are  consistent  with  what  we  reported  in  our  prior  assessment. 


Factors  That  Explain  the 
Changes  in  the  2015 
Portfolio 


4.  Older  programs  in  the  portfolio  carry  a  majority  of  the  total  cost 
and  cost  growth  since  first  full  estimates  compared  to  newer 
programs.  Of  the  79  programs  in  the  2015  portfolio,  40  were  also 
in  the  2005  portfolio  and  represent  80  percent  of  the  portfolio’s 
total  acquisition  cost — over  $1.1  trillion  of  the  $1.4  trillion  total. 
The  40  programs  common  to  the  2005  and  2015  portfolios  range  in 
age  between  1 1  and  38  years.  These  programs  account  for  80 
percent  of  the  current  portfolio’s  $1.4  trillion  in  total  acquisition  cost. 
Consequently,  the  40  newer  programs  account  for  just  20  percent  of 
the  total  portfolio  acquisition  cost. 

The  total  acquisition  cost  of  the  portfolio  is  also  driven  in  large  part  by 
the  10  costliest  programs,  which  currently  account  for  $866  billion,  or 
60  percent  of  the  portfolio’s  total  cost.  Nine  of  the  10  costliest 
programs  in  the  2015  portfolio  were  also  in  the  2005  portfolio.  These 
programs  are  over  1 1  years  old  and  account  for  65  percent  of  the 
portfolio’s  cost  growth  since  first  full  estimates  as  shown  in  table  3 
below. 


8When  calculating  this  delay,  we  obtained  schedule  information  for  the  cycle  time  from 
program  start  to  initial  operational  capability  as  reported  in  the  previous  year  and 
contrasted  it  with  the  current  schedule. 

9The  first  full  estimate  is  generally  the  cost  estimate  established  at  the  start  of  system 
development,  for  more  information  see  Appendix  I.  For  more  information  on  the  portfolio’s 
performance  since  first  full  estimates,  see  Appendix  III. 
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Table  3:  Programs  with  the  Largest  Total  Acquisition  Cost  in  the  2015  Portfolio 
Fiscal  year  2016  dollars  (in  billions) 


Program 

Years  since  first 
full  estimate 

Total  acquisition 
cost 

Total  acquisition  cost  growth 
since  initial  estimates 

F-35  Joint  Strike  Fighter 

14 

$340 

$111 

DDG  51  Arleigh  Burke  Class  Guided  Missile  Destroyer 

33 

115 

99 

SSN  774  Virginia  Class  Submarine 

21 

91 

26 

V-22  Osprey  Joint  Services  Advanced  Vertical  Lift  Aircraft 

33 

62 

19 

Evolved  Expendable  Launch  Vehicle 

19 

61 

42 

Trident  II  Missile 

38 

58 

2 

KC-46  Tanker  Modernization  Program 

4.9 

44 

-4 

Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier 

11.7 

36 

-2 

P-8A  Poseidon  Multi-Mission  Maritime  Aircraft 

15.8 

33 

.03 

UH-60M  Black  Hawk  Helicopter 

14.7 

26 

12 

Ten  costliest  programs 

$866 

$306 

Remaining  portfolio  total 

$574 

$163 

2015  portfolio  total 

$1,440 

$469 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 

Eighty  percent  of  the  2015  portfolio’s  cost  growth  since  first  full 
estimates — $373  billion — occurred  five  or  more  years  ago.  Since 
2010,  the  current  portfolio  has  accumulated  $95.7  billion  in  cost 
growth,  about  20  percent  of  the  $469  billion  total.  That  same  year, 
DOD  began  implementation  of  WSARA  and  its  “Better  Buying  Power” 
initiatives.10  In  addition,  our  analysis  also  shows  that  programs  started 
since  WSARA  and  the  “Better  Buying  Power”  initiatives  have  realized 
cost  decreases  from  their  first  full  estimates.  The  14  programs  that 
began  system  development  since  2010  have  collectively  reduced  their 
total  estimated  acquisition  costs  by  over  $580  million  since  their  first 
full  estimates.  However,  these  programs  have  only  recently  begun 
development  and  may  encounter  issues  in  the  future.  DOD  and 
Congress  will  need  to  continue  oversight  to  ensure  they  do  not 
increase  in  cost. 


10Pub.  L.  No.  111-23. 
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5.  Of  the  79  programs,  37  saw  $19.3  billion  in  cost  decreases  while 
42  programs  experienced  $16.8  billion  in  cost  increases  resulting 
in  a  net  decrease  in  the  total  acquisition  cost  of  nearly  $2.5 
billion.  The  $2.5  billion  decrease  shown  in  table  2  above  is  the  net 
result  of  cost  changes  on  all  79  programs  in  the  current  portfolio.  The 
distribution  of  those  cost  changes  across  the  entire  portfolio  is  shown 
below  in  figure  3. 


Figure  3:  Distribution  of  the  1-year  Change  in  Total  Acquisition  Cost  within  the  2015  Portfolio 


Fiscal  year  2016  dollars  (in  billions) 


Number  of  programs 


-10 

<-25%  -25-20%  -20-15%  -15-10%  -10-5%  -5-0%  0-5% 


5-10%  10-15%  15-20%  20-25%  >25% 


Percentage  cost  change  interval 


Amount  of  cost  change  in  each  percentage  interval  over  the  past  year 
Number  of  programs  in  each  percentage  cost  change  interval 
Source:  GAO  analysis  of  DOD  data  |  GAO-16-329SP 
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Of  the  37  programs  that  decreased  their  total  acquisition  cost  over  the 
past  year, 

•  29  did  so  by  finding  efficiencies  in  the  program  and  not  by 
changing  procurement  quantities  resulting  in  total  cost  decreases 
of  $9.5  billion; 

•  6  reduced  their  planned  procurement  quantities  thereby  reducing 
overall  cost  by  $8.1  billion;  and 

•  2  programs — the  Evolved  Expendable  Launch  Vehicle  and  the 
Ship-to-Shore  Connector  Amphibious  Craft — were  able  to  reduce 
their  overall  cost  by  $1.7  billion  while  increasing  their  planned 
procurement  quantities. 

In  contrast,  42  programs  reported  total  acquisition  cost  increases  over 
the  past  year.  Of  these, 

•  27  reported  cost  increases  due  to  inefficiencies  in  the  program 
rather  than  a  change  in  procurement  quantities  resulting  in  a  total 
cost  increase  of  almost  $2.9  billion; 

•  15  increased  their  planned  procurement  quantities  resulting  in  a 
cost  increase  of  almost  $14  billion. 

6.  Thirty-eight  programs  in  the  portfolio  gained  buying  power 

during  the  past  year  resulting  in  a  gain  of  $10.7  billion.  There  are 
a  few  programs  that  gain  or  lose  buying  power  year  after  year;  6 
programs  in  our  analysis  report  such  gains  or  losses.  As  can  be 

seen  from  the  preceding  discussion  of  individual  cost  increases  and 
decreases,  to  better  understand  the  changes  in  the  portfolio’s  total 
cost  over  the  past  year,  the  effect  of  changes  in  quantity  on  individual 
programs  must  be  analyzed  and  understood.  In  general,  buying  power 
can  be  defined  as  the  amount  of  goods  or  services  that  can  be 
purchased  given  a  specified  level  of  funding.  To  determine  changes  in 
buying  power,  the  effects  of  quantity  changes  must  be  isolated  from 
other  factors  that  affect  cost.  For  example,  a  program’s  cost  can 
increase  solely  because  of  additional  quantities.  While  that  does 
represent  a  cost  increase,  it  does  not  necessarily  indicate  acquisition 
problems  or  a  loss  of  buying  power.  Alternatively  a  program’s  cost  can 
decrease  due  to  a  reduction  in  quantity  and  may  still  experience  a 
buying  power  gain  or  loss. 

According  to  our  calculation  of  cost  changes  attributable  to  quantity 
changes  we  would  have  expected  the  portfolio  to  increase  by  $4.7 
billion  in  procurement  costs.  Instead,  procurement  costs  for  the 
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portfolio  have  decreased  by  $6  billion  over  the  past  year,  indicating 
that  programs  found  efficiencies  in  other  areas  to  offset  the  cost  of 
additional  quantities  and  the  portfolio  overall  realized  a  buying  power 
gain.11  Our  calculation  of  how  programs’  cost  and  quantity  changes 
affected  their  buying  power  is  presented  in  table  4. 


Table  4:  Buying  Power  Analysis  for  the  2015  Portfolio 

Fiscal  year  2016  dollars  (in  billions) 

Number 

of 

programs 

GAO-calculated 
cost  change 
attributable  to 
quantity  changes 

Actual 
procurement 
cost  change 

GAO  calculated 
cost  change  not 
attributable  to 
quantity 
changes 

Programs  that  gained  buying  power 

38 

$10.6 

-$5.4 

-$16.0 

Procurement  cost  decreased  with  no  quantity  change 

26 

$0.0 

-$10.6 

-$10.6 

Quantity  increased  with  less  cost  increase  than  anticipated 

11 

$10.6 

$5.9 

-$4.7 

Quantity  decreased  with  more  cost  decrease  than  anticipated 

1 

-$.06 

-$0.7 

-$0.7 

Programs  that  lost  buying  power 

35 

-$4.2 

$1.1 

$5.3 

Procurement  cost  increased  with  no  quantity  change 

25 

$0.0 

$2.0 

$2.0 

Quantity  increased  with  more  cost  increase  than  anticipated 

6 

$4.9 

$5.5 

$0.6 

Quantity  decreased  with  less  cost  decrease  than  anticipated 

4 

-$9.1 

-$6.3 

$2.8 

Programs  with  no  change  in  buying  power 

5 

$0.0 

$0.0 

$0.0 

Program  eliminated  procurement 

1 

-$1.7 

-$1.7 

$0.0 

Portfolio  totals 

79 

$4.7 

-$6.0 

-$10.7 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 

Note:  Some  numbers  may  not  sum  due  to  rounding. 

Our  analysis  shows  that  38  programs  increased  their  buying  power  in 
the  past  year  and  reduced  procurement  costs  by  a  total  of  $5.4  billion. 
This  total  is  the  net  amount  of  cost  change  given  increases  and 
decreases  due  to  other  program  efficiencies.  Twenty-six  of  these  38 
programs  decreased  procurement  costs  with  no  change  in  their 
procurement  quantity  indicating  that  they  found  efficiencies 
elsewhere.  Some  of  these  programs  reported  identifying  significant 
realized  and  expected  “should-cost”  savings  that  resulted  in  a 
reduction  of  procurement  cost  or  offset  the  cost  of  additional 
quantities. 


description  of  the  calculation  used  can  be  found  in  appendix  I. 
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Eleven  programs  are  buying  additional  quantities  at  lower  prices.  In 
other  words,  their  planned  procurement  quantities  increased  but  the 
corresponding  procurement  cost  did  not  increase  in  kind  or  was  offset 
by  other  efficiencies.  For  example,  the  SSN  774  Virginia  Class 
Submarine  procured  two  additional  submarines  with  a  lower  than 
expected  procurement  cost  increase.  Our  analysis  indicates  that  if 
only  the  quantity  increase  is  considered,  the  program’s  procurement 
costs  should  have  risen  by  $5.2  billion.  Instead,  the  program  realized 
a  buying  power  gain,  due  to  program  cost  efficiencies  and  realized 
reductions  of  $390  million. 

Thirty-five  programs  lost  buying  power  in  the  past  year  with  actual 
procurement  cost  increases  of  $1 .1  billion.  By  our  calculations,  the  net 
result  of  quantity  changes  on  these  programs  should  have  resulted  in 
a  $4.2  billion  cost  decrease  due  to  reductions  in  quantities  on  4  of 
these  35  programs.  This  means  that  procurement  cost  increases  not 
related  to  quantity  changes  generated  $5.3  billion  in  additional  costs 
and  a  net  buying  power  loss.  Contributing  to  this  were  25  programs 
that  lost  buying  power  as  their  procurement  costs  increased  with  no 
change  in  quantities,  an  indication  of  inefficiencies  within  these 
programs.  For  example,  the  CH-47F  Improved  Cargo  Helicopter 
program  lost  buying  power  as  it  experienced  a  procurement  cost 
increase  of  $39  million  over  the  past  year  with  no  change  in  its 
planned  procurement  quantity.  Six  other  programs  increased  their 
planned  procurement  quantities  but  incurred  a  higher  than  expected 
procurement  cost  increase,  indicating  that  they  lost  efficiencies 
elsewhere.  The  remaining  4  programs  reported  decreases  in  their 
initial  quantities;  however,  the  cost  reductions  on  these  programs  are 
less  than  expected  by  our  calculations. 

In  addition,  there  are  a  handful  of  individual  programs  that  gain  or  lose 
buying  power  consistently: 

•  Three  programs — Tactical  Tomahawk,  MQ-4C  Triton  Unmanned 
Aircraft  System,  and  C-5  Reliability  Enhancement  and  Re¬ 
engineering  Program — experienced  gains  in  buying  power  each 
year  over  the  past  5  years.  Tactical  Tomahawk  changed  its 
quantities  in  3  of  the  past  5  years  but  quantities  for  the  other 
programs  did  not  change. 

•  Three  programs — CH-53K  Heavy  Lift  Replacement  Helicopter, 
DDG  1000  Zumwalt  Class  Destroyer,  and  Joint  Precision 
Approach  and  Landing  System — lost  buying  power  in  each  of  the 
past  5  years.  These  programs  generally  had  successive  years  of 
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procurement  cost  increases  with  only  minimal  quantity  changes 
that  were  outweighed  by  other  inefficiencies. 

Of  the  programs  that  have  consistently  lost  buying  power  over  the 
past  5  years,  most  have  also  had  schedule  delays  longer  than  those 
that  gained  buying  power  consistently. 

7.  Schedule  delays  over  the  past  year  in  1 1  of  the  79  programs 
contributed  to  the  portfolio’s  overall  delay  of  2.4  months.  The 
Airborne  Maritime/Fixed  Station  radio  system  experienced  the 
largest  delay  at  more  than  8  years,  which  disproportionately 
affected  the  portfolio.  Thirteen  programs  in  the  current  portfolio 
reported  a  schedule  change  over  the  past  year,  1 1  of  which  were 
delays.  The  average  2.4  month  schedule  increase  for  2015  is  the 
result  of  changes  reported  by  all  the  programs  in  the  current  portfolio. 
If  the  effect  of  the  Airborne  Maritime/Fixed  Station  radio  system’s 
delay  is  removed  from  the  analysis,  the  average  schedule  increase  for 
2015  is  1.1  months,  similar  to  what  we  reported  in  our  last 
assessment.12  Figure  4  below  shows  the  schedule  changes  over  the 
past  year. 


12GAO,  Defense  Acquisitions:  Assessments  of  Selected  Weapon  Programs , 
GAO-1 5-342SP  (Washington,  D.C.:  Mar.  12,  2015). 
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Figure  4:  Distribution  of  the  1-year  Change  in  Delivery  of  Initial  Operational 
Capability  within  the  2015  Portfolio 
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Source:  GAO  analysis  of  DOD  data.  |  GAO-16-329SP 


Six  programs  reported  schedule  delays  of  more  than  6  months  over 
the  past  year.  For  example,  due  to  an  overall  Joint  Tactical  Radio 
System  reorganization,  the  Airborne  Maritime/Fixed  Station  radio 
system  was  restructured  in  2012.  The  program  had  not  been  reporting 
an  initial  operational  capability  delivery  date  until  this  past  year  when  it 
reported  a  delay  of  98  months,  more  than  8  years,  from  its  initial 
estimate.  In  another  case,  the  Next  Generation  Operational  Control 
System  reported  a  delay  of  27  months  over  the  past  year.  Due  to 
issues  with  the  integration  and  testing  of  its  initial  block  of  capability 
and  the  concurrent  systems  engineering  approach  for  subsequent 
blocks,  the  program  reported  delays  to  its  delivery  of  initial  capability 
and  its  production  decision  dates.  Similarly,  the  DDG  1000  Zumwalt 
Class  Destroyer  reported  a  delay  of  26  months  over  the  past  year  due 
to  technical  risk,  shipyard  performance,  and  shipyard  workforce 
constraints.  Another  5  of  the  1 1  programs  reported  schedule  delays  of 
less  than  six  months  over  the  past  year. 
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Two  programs — the  B-2  Extremely  High  Frequency  Satellite 
Communications  and  Computer  Increment  1  and  Space  Fence 
Ground-Based  Radar  System  Increment  1 — reported  dates  for 
delivery  of  initial  operational  capability  3  and  9  months  earlier  than 
expected.  The  B-2  Extremely  High  Frequency  Satellite 
Communications  and  Computer  Increment  1  date  was  revised  to 
reflect  the  actual  delivery  of  capability  and  the  Space  Fence 
program’s  change  is  the  result  of  a  schedule  revision  post-contract 
award. 

8.  Sixteen  programs  in  production  reported  development  cost 
growth  over  the  past  year  even  though  production  is  a  phase  of 
the  acquisition  cycle  which  should  have  minimal  development 
cost  growth.  This  represents  concurrency,  albeit  for  differing 
reasons,  and  is  a  contributor  to  cost  growth.  The  16  programs  in 
our  assessment  that  report  development  cost  increases  of  2  percent 
or  more  over  the  past  year  account  for  $3.3  billion  in  development 
cost  growth.  These  programs  are  in  production,  a  phase  of  the 
acquisition  cycle  which  should  have  minimal  development  cost 
growth.  On  average,  these  16  programs  have  been  in  the  portfolio 
more  than  16  years  and  56  percent  of  them  have  already  achieved 
their  initial  operating  capability.  Table  5  shows  the  extent  and  causes 
of  the  development  cost  growth  on  these  programs. 


Table  5:  Programs  with  the  Largest  Development  Cost  Percentage  Increases  over  the  Past  Year 

Fiscal  year  2016  dollars  (in  millions) 

Program 

Percentage 
increase  in 
development 
cost  over  the 
past  year 

Amount  of 
development 
cost  growth  over 
the  past  year 

Initial 

capability 

achieved 

Primary  cause  for 
development  cost 
increase 

AIM-9X  Block  II  Air-to-Air  Missile 

45% 

$172 

No 

Deficiency 

MQ-8  Fire  Scout 

36 

325 

Yes 

Unplanned  capability 

Evolved  Expendable  Launch  Vehicle 

21 

528 

Yes 

Unplanned  capability 

Navy  Multiband  Terminal 

18 

135 

Yes 

Unplanned  capability 

Patriot  Advanced  Capability-3  Missile  Segment  Enhancement 

9 

80 

No 

Unplanned  capability 

Family  of  Advanced  Beyond-Line-of-Sight  Terminals 

9 

215 

No 

Deficiency 

DDG  51  Arleigh  Burke  Class  Guided  Missile  Destroyer 

6 

364 

Yes 

Unplanned  capability 

Global  Positioning  System  III 

6 

180 

NA 

Deficiency 

Next  Generation  Operational  Control  System 

5 

190 

No 

Deficiency 

LHA  6  America  Class  Amphibious  Assault  Ship 

4 

17 

No 

Unplanned  capability 

Littoral  Combat  Ship  Mission  Packages 

4 

93 

No 

Deficiency 

EA-18G  Growler  Aircraft 

4 

85 

Yes 

Unplanned  capability 
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Program 

Percentage 
increase  in 
development 
cost  over  the 
past  year 

Amount  of 
development 
cost  growth  over 
the  past  year 

Initial 

capability 

achieved 

Primary  cause  for 
development  cost 
increase 

AIM-120  Advanced  Medium  Range  Air-to-Air  Missile 

4 

157 

Yes 

Unplanned  capability 

V- 22  Osprey  Joint  Services  Advanced  Vertical  Lift  Aircraft 

3 

484 

Yes 

Unplanned  capability 

Multifunctional  Information  Distribution  System 

3 

69 

Yes 

Unplanned  capability 

Cooperative  Engagement  Capability 

3 

109 

Yes 

Unplanned  capability 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 


Development  cost  increases  on  these  programs  are  generally  due  to 
one  of  two  factors: 

•  Five  of  the  1 6  needed  additional  funding  to  correct  for  deficiencies 
found  in  testing.  For  example,  the  AIM-9X  Block  II  program 
reported  a  development  cost  increase  of  45  percent  due  to 
deficiencies  found  during  operational  testing. 

•  Eleven  of  the  1 6  added  unplanned  capability  to  a  program’s 
baseline.  Increasing  the  capabilities  of  a  program  beyond  the 
established  requirements  adds  cost  and  potential  risk  to  mature 
weapon  systems  already  in  production.  Three  of  the  four  largest 
development  percentage  increases  are  due  to  increases  in 
capability.  Adding  capability  while  in  production  leads  to 
concurrency  in  development  and  production  and  can  lead  to 
instability.  In  some  cases  the  added  capability  may  be  warranted 
whereas  in  others  it  might  be  best  to  develop  and  deliver  a  new 
capability  incrementally,  which  could  be  part  of  a  new  major 
defense  acquisition  program,  rather  than  as  an  add-in  to  a  current 
program,  as  recommended  by  best  practices  for  cost  estimating.13 
Adding  capability  to  existing  cost  baselines  can  also  create 
oversight  challenges  as  it  becomes  difficult  to  differentiate  the 
reasons  for  cost  growth. 


Other  Observations  9.  As  measured  against  the  metrics  discussed  with  the  Office  of 

Management  and  Budget  (OMB)  and  DOD,  more  programs  meet 
each  metric  for  cost  change  than  previous  assessments.  Most 
notably,  72  percent  of  programs  meet  the  threshold  for  less  than 


13GAO,  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Developing  and 
Managing  Capital  Program  Costs,  GAO-09-3SP  (Washington,  D.C.:  Mar.  2,  2009). 
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10  percent  growth  over  the  past  5  years  compared  to  the  2013 
portfolio  in  which  55  percent  of  programs  met  this  metric. 
Seventy-six  percent  meet  the  threshold  for  less  than  2  percent 
growth  in  the  past  year.  Forty-seven  percent  meet  the  threshold 
for  less  than  15  percent  cost  growth  since  first  full  estimates.  In 
December  2008,  GAO,  OMB,  and  DOD  discussed  a  set  of  outcome 
metrics  and  goals  to  measure  program  cost  performance  over  time. 
The  metrics  are  intended  to  measure  cost  performance  on  a 
percentage  basis  over  three  defined  periods:  the  preceding  year,  the 
preceding  5  years,  and  since  first  full  estimates  were  established.14 
We  have  reported  on  these  outcomes  since  2012  and  figure  5  shows 
how  the  performance  of  the  current  portfolio  compares  to  our  prior 
assessments. 


14DOD  no  longer  supports  the  use  of  these  metrics.  We  continue  to  believe  that  they  have 
value. 
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Figure  5:  Comparison  of  the  Cost  Performance  of  DOD’s  2011-2015  Portfolios 
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Note:  For  the  2014  Portfolio  we  did  not  calculate  a  5-year  comparison  as  there  were  no  December 
2008  SARs  issued. 


Seventy-two  percent  of  programs  in  the  current  portfolio  kept  their 
cost  growth  under  10  percent  over  the  past  5  years.  This  is  a 
significant  improvement  compared  to  the  2013  portfolio,  the  last  time 
we  assessed  this  metric.  As  this  metric  measures  programs’  cost 
growth  over  the  past  5  years,  the  improvement  in  the  number  of 
programs  meeting  this  metric  may  be  due  to  programs  started  since 
2010  and  the  department’s  implementation  of  WSARA  and  the  “Better 
Buying  Power”  initiatives.  Programs  started  before  2010  have 
experienced  over  $469  billion  in  total  acquisition  cost  growth  since 
their  first  full  estimates  whereas  programs  started  after  2010  have 
experienced  over  $580  million  in  decreases.  However,  these 
programs  have  only  recently  begun  development  and  will  need 
continued  oversight  to  ensure  they  do  not  increase  in  cost  in  the 
future. 

Seventy-six  percent  of  programs  in  the  2015  portfolio  meet  the  one 
year  cost  performance  metric  by  limiting  total  acquisition  cost  growth 
to  less  than  2  percent,  and  47  percent  of  programs  meet  the  threshold 
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for  less  than  15  percent  cost  growth  since  first  full  estimates.  Although 
a  larger  share  of  programs  is  meeting  the  1-year  metric  than  in  our 
last  assessment,  performance  against  the  first  full  estimate  cost 
growth  threshold  is  relatively  unchanged,  remaining  less  than  50 
percent. 

10.  Army  programs’  percentage  of  the  total  cost  of  the  major 
defense  acquisition  program  portfolio  is  the  smallest  of  the 
services  and  has  been  decreasing  since  2007.  The  percentage 
attributable  to  the  Navy  and  Air  Force  programs’  have  been 
increasing  since  2007  and  2012  respectively.  Navy  programs 
account  for  almost  55  percent  of  the  2015  total  estimated 
acquisition  cost.  The  79  programs  in  the  current  portfolio  include 
three  “joint”  or  DOD-wide  programs — the  F-35  Joint  Strike  Fighter,  the 
Joint  Light  Tactical  Vehicle,  and  Chemical  Demilitarization — 15  Army 
programs,  26  Air  Force  programs,  and  35  Navy  programs.  The  Army 
accounts  for  almost  10  percent,  the  Air  Force  accounts  for  over  34 
percent,  and  the  Navy  accounts  for  almost  55  percent  of  the  portfolio’s 
total  acquisition  cost  in  2015,  including  the  service-specific  funding 
within  joint  programs.  Since  2005,  the  Army’s  portion  of  the  portfolio 
has  decreased,  the  Air  Force’s  portion  has  increased  slightly,  and  the 
Navy’s  portion  has  increased  to  its  current  level  of  55  percent  from  a 
low  of  43  percent  in  2007.  Figure  6  shows  each  service’s  portion  of 
the  portfolio’s  total  acquisition  cost  since  2005. 
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Figure  6:  Services’  Percentage  of  the  Portfolio’s  Total  Acquisition  Cost,  2005-2015 


Percentage  of  Total  Acquisition  Cost  by  component 
0.7  0.7  0.8  4.4  0.8 


100 


80 


60 


40 


20 


23.5 


25.5 


1.0  1.1  1.1  1.4  1.4 


26.9 


30.4  ■  29.2  ■  29.5  ■  ■  29.8  ■  28.5  ■  32.3  ■  33.2  ■  34.3 


16.2 


16.1 


12.2 


11.5 


9.8 


Office  of  the  Secretary  of  Defense 
Army 


Air  Force 


45.4  ■  44.7  ■  42.8  ■  43.8  ■  49.6  ■  53.0  ■  54.2  ■  54.4  ■  53.9  ■  54.6 


Navy 


2005  2006  2007  2008  //  2010  2011  2012  2013  2014  2015 

Fiscal  year 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 


In  a  constrained  funding  environment,  unforeseen  cost  growth  limits 
investment  choices  within  DOD.  The  Navy  has  historically  had  more 
programs  than  the  Air  Force  and  Army.  Currently,  the  Navy  has  over 
30  programs  in  the  portfolio,  while  the  Army  has  16  programs 
including  the  Joint  Light  Tactical  Vehicle.  When  looking  at  total  cost, 
the  Navy  currently  accounts  for  $785  billion  of  the  $1 .4  trillion  total 
acquisition  cost.  This  cost  can  be  expected  to  grow  as  the  Navy  does 
not  include  all  of  the  anticipated  ships  of  a  class  in  the  Selected 
Acquisition  Reports  (SAR)  we  use  to  assess  programs.  For  example, 
while  there  may  be  up  to  1 1  Ford-class  aircraft  carriers  delivered,  the 
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current  cost  reporting  only  accounts  for  3.  We  have  reported  on  many 
of  the  Navy’s  programs  and  their  acquisition  challenges.15 

11.  The  equity  prices  of  the  contractors  delivering  the  10  costliest 
programs  have  performed  well  relative  to  broad-based  market 
indices,  indicating  that  investors  expect  these  firms  to  remain 
profitable  well  into  the  future.  Five  publicly-traded  defense 
contractors  are  developing  and  delivering  the  10  largest  DOD 
programs  in  the  2015  portfolio.  The  equity  prices — a  stock  price- 
based  indicator  of  investor  expectations  of  future  earnings  over  many 
years — for  these  contractors  over  the  past  decade  has  increased  at  a 
rate  that  outperformed  broad-based  indices  for  markets  as  a  whole, 
as  well  as  narrower  indices  covering  similar  sectors  of  the  economy.16 
This  indicates  that  investors  expect  the  performance  of  these 
companies  to  be  particularly  strong  for  some  time  to  come.  Strong 
equity  performance  among  these  contractors  could  be  driven  by  a 
number  of  factors  and  is  not  necessarily  related  to  the  financial 
condition  of  major  acquisition  programs. 


15GAO,  Ford  Class  Aircraft  Carrier:  Poor  Outcomes  Are  the  Predictable  Consequences  of 
the  Prevalent  Acquisition  Culture,  GAO-16-84T  (Washington,  D.C.:  Oct.  1,  2015);  Ford- 
Class  Aircraft  Carrier:  Congress  Should  Consider  Revising  Cost  Cap  Legislation  to 
Include  All  Construction  Costs,  GAO-15-22  (Washington,  D.C.:  Nov.  20,  2014);  Littoral 
Combat  Ship:  Navy  Complied  with  Regulations  in  Accepting  Two  Lead  Ships,  but  Quality 
Problems  Persisted  after  Delivery,  GAO-14-827  (Washington,  D.C.:  Sept.  25,  2014);  Navy 
Shipbuilding:  Significant  Investments  in  the  Littoral  Combat  Ship  Continue  Amid 
Substantial  Unknowns  about  Capabilities,  Use,  and  Cost,  GAO-13-530  (Washington,  D.C.: 
July  22,  2013);  Arleigh  Burke  Destroyers:  Additional  Analysis  and  Oversight  Required  to 
Support  the  Navy’s  Future  Surface  Combatant  Plans,  GAO-12-113  (Washington,  D.C.: 
Jan.  24,  2012);  V-22  Osprey  Aircraft:  Assessments  Needed  to  Address  Operational  and 
Cost  Concerns  to  Define  Future  Investments,  GAO-09-692T  (Washington,  D.C.:  June  23, 
2009);  Defense  Acquisitions:  Realistic  Business  Cases  Needed  to  Execute  Navy 
Shipbuilding  Programs,  GAO-07-943T  (Washington,  D.C.  July  24,  2007);  Multiyear 
Procurement  Authority  for  the  Virginia  Class  Submarine  Program,  GAO-03-895R: 
(Washington,  D.C.:  June  23,  2003). 

16lndices  used  are  based  on  Standard  &  Poor’s  Global  Industry  Classification  System. 
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Observations  from 
Our  Assessment  of 
Knowledge  Attained 
by  Programs  at  Key 
Junctures 


Our  analysis  found  that  while  a  few  programs  are  using  knowledge-based 
approaches  to  reduce  their  acquisition  risk,  implementation  of  these 
approaches  across  the  portfolio  has  been  uneven.  For  example, 
programs  that  recently  began  system  development  demonstrated  some 
best  practices — such  as  holding  a  preliminary  design  review  prior  to 
committing  resources.  Others,  however,  are  carrying  technology  risk  well 
into  system  development,  failing  to  demonstrate  designs  through 
prototyping,  or  proceeding  into  production  before  ensuring  manufacturing 
processes  are  under  control  and  testing  is  complete. 


Our  body  of  work  has  shown  that  positive  acquisition  outcomes  require 
the  use  of  a  knowledge-based  approach  to  product  development  that 
demonstrates  high  levels  of  knowledge  before  significant  commitments 
are  made.  In  essence,  knowledge  supplants  risk  overtime.  In  our  past 
work  examining  weapon  acquisition  and  best  practices  for  product 
development,  we  have  found  that  leading  commercial  firms  and 
successful  DOD  programs  pursue  an  acquisition  approach  that  is 
anchored  in  knowledge,  whereby  high  levels  of  product  knowledge  are 
demonstrated  at  critical  points  in  the  acquisition  process.  This  work  led  to 
multiple  recommendations  that  DOD  generally  or  partially  agreed  with 
and  has  made  progress  in  implementing.17  On  the  basis  of  this  work,  we 
have  identified  three  key  knowledge  points  during  the  acquisition  cycle — 
development  start,  system-level  critical  design  review,  and  production 
start — at  which  programs  need  to  demonstrate  critical  levels  of  knowledge 
to  proceed.  Figure  7  aligns  the  acquisition  milestones  described  in  DOD’s 
primary  acquisition  policy  with  these  knowledge  points.  In  this  report,  we 
refer  to  DOD’s  engineering  and  manufacturing  development  phase  as 
system  development.  Production  start  typically  refers  to  a  program’s  entry 
into  low-rate  initial  production. 


17GAO,  Best  Practices:  DOD  Can  Achieve  Better  Outcomes  by  Standardizing  the  Way 
Manufacturing  Risks  Are  Managed,  GAO-10-439  (Washington,  D.C.:  Apr.  22,  2010);  Best 
Practices:  High  Levels  of  Knowledge  at  Key  Points  Differentiate  Commercial  Shipbuilding 
from  Navy  Shipbuilding,  GAO-09-322  (Washington,  D.C.:  May  13,  2009);  Defense 
Acquisitions:  A  Knowledge-Based  Funding  Approach  Could  Improve  Major  Weapon 
System  Program  Outcomes,  GAO-08-619  (Washington,  D.C.:  July  2,  2008);  Best 
Practices:  Capturing  Design  and  Manufacturing  Knowledge  Early  Improves  Acquisition 
Outcomes,  GAO-02-701  (Washington,  D.C.:  July  15,  2002);  Best  Practices:  Better 
Matching  of  Needs  and  Resources  Will  Lead  to  Better  Weapon  System  Outcomes, 
GAO-01-288  (Washington,  D.C.:  Mar.  8,  2001);  and  Best  Practices:  Better  Management  of 
Technology  Development  Can  Improve  Weapon  System  Outcomes,  GAO/NSIAD-99-162 
(Washington,  D.C.:  July  30,  1999). 
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Figure  7:  DOD’s  Acquisition  Cycle  and  GAO  Knowledge  Points 
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Source:  GAO.  |  GAO-1 6-329SP 


As  our  prior  work  has  shown,  the  building  of  knowledge  consists  of 
information  that  should  be  gathered  at  three  critical  points  over  the  course 
of  a  program. 

Knowledge  point  1:  Resources  and  requirements  match.  Achieving  a 
high  level  of  technology  maturity  by  the  start  of  system  development  is 
one  of  several  important  indicators  of  whether  this  match  has  been  made. 
This  means  that  the  technologies  needed  to  meet  essential  product 
requirements  have  been  demonstrated  to  work  in  a  relevant  environment. 
In  addition,  the  developer  should  complete  a  series  of  systems 
engineering  reviews  culminating  in  a  preliminary  design  of  the  product 
that  shows  the  design  is  feasible.  Constraining  the  development  phase  of 
a  program  to  5  or  6  years  is  also  recommended  because  it  aligns  with 
DOD’s  budget  planning  process  and  fosters  the  negotiation  of  trade-offs 
in  requirements  and  technologies.  For  shipbuilding  programs,  critical 
technologies  should  be  matured  into  actual  sub-system  prototypes  and 
successfully  demonstrated  in  an  operational  environment  before  a 
contract  is  awarded  for  the  detailed  design  of  a  new  ship. 

Knowledge  point  2:  Product  design  is  stable.  This  point  occurs  when  a 
program  determines  that  a  product’s  design  will  meet  customer 
requirements,  as  well  as  cost,  schedule,  and  reliability  targets.  A  best 
practice  is  to  achieve  design  stability  at  the  system-level  critical  design 
review,  usually  held  midway  through  system  development.  Completion  of 
at  least  90  percent  of  engineering  drawings  at  this  point  provides  tangible 
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evidence  that  the  product’s  design  is  stable,  and  a  prototype 
demonstration  shows  that  the  design  is  capable  of  meeting  performance 
requirements.  Shipbuilding  programs  should  demonstrate  design  stability 
by  completing  100  percent  of  the  basic  and  functional  drawings,  as  well 
as  the  three-dimensional  product  model  by  the  start  of  construction  for  a 
new  ship.  Programs  can  also  improve  the  stability  of  their  design  by 
conducting  reliability  growth  testing  and  completing  failure  modes  and 
effects  analyses  so  fixes  can  be  incorporated  before  production  begins. 

At  this  point,  programs  should  also  begin  preparing  for  production  by 
identifying  manufacturing  risks,  key  product  characteristics,  and  critical 
manufacturing  processes. 

Knowledge  point  3:  Manufacturing  processes  are  mature.  This  point 
is  achieved  when  it  has  been  demonstrated  that  the  developer  can 
manufacture  the  product  within  cost,  schedule,  and  quality  targets.  A  best 
practice  is  to  ensure  that  all  critical  manufacturing  processes  are  in 
statistical  control — that  is,  they  are  repeatable,  sustainable,  and  capable 
of  consistently  producing  parts  within  the  product’s  quality  tolerances  and 
standards — at  the  start  of  production.  Demonstrating  critical  processes  on 
a  pilot  production  line  is  an  important  initial  step  in  this  effort.  In  addition, 
production  and  postproduction  costs  are  minimized  when  a  fully 
integrated,  capable  production-representative  prototype  is  demonstrated 
to  show  that  the  system  will  work  as  intended  in  a  reliable  manner  before 
committing  to  production.  We  did  not  assess  shipbuilding  programs  for 
this  knowledge  point  due  to  differences  in  the  production  processes  used 
to  build  ships. 

Knowledge  in  these  three  areas  builds  over  time.  Our  prior  work  on 
knowledge-based  approaches  shows  that  a  knowledge  deficit  early  in  a 
program  can  cascade  through  design  and  production,  leaving  decision 
makers  with  less  knowledge  to  support  decisions  about  when  and  how  to 
best  move  into  subsequent  acquisition  phases  that  commit  more 
budgetary  resources.  Demonstrating  technology  maturity  is  a  prerequisite 
for  moving  forward  into  system  development,  during  which  time  the  focus 
should  be  on  design  and  integration.  A  stable  and  mature  design  is  also  a 
prerequisite  for  moving  forward  into  production,  where  the  focus  should 
be  on  efficient  manufacturing.  Additional  details  about  key  practices  at 
each  of  the  knowledge  points  can  be  found  in  appendix  IV. 
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For  this  report,  we  assessed  the  knowledge  attained  at  key  junctures  in 
the  acquisition  process  for  43  current  programs,  most  of  which  are  in 
development  or  early  production.18  Not  all  programs  included  in  our 
review  of  knowledge-based  practices  provided  information  for  every 
knowledge  point  and  some  had  not  reached  all  of  the  knowledge  points — 
development  start,  design  review,  and  production  start — at  the  time  of  this 
assessment.  We  also  reviewed  the  knowledge  that  12  future  major 
defense  acquisition  programs  identified  by  DOD  expect  to  attain  when 
they  start  system  development  in  the  coming  years.19 

Our  analysis  of  the  data  from  these  current  and  future  programs  allows  us 
to  make  the  following  three  observations. 


Knowledge  Point  Observations 

1 .  Of  the  seven  programs  that  began,  or  are  planning  to  begin,  system  development  during  our 
assessment  period,  none  have  demonstrated  a  full  match  between  resources  and 
requirements.  One  program  plans  to  enter  system  development  with  immature  technologies 
whereas  the  rest  will  have  technologies  approaching  full  maturity.  Five  of  the  seven 
programs  completed  a  preliminary  design  review  before  development  start.  Two  did  not 
conduct  all  of  the  early  system  engineering  reviews  recommended  by  best  practices.  All  but 
one  program  plan  to  constrain  their  development  phase  to  6  six  years  or  less.  The 
implementation  of  best  practices  by  these  programs  is  better  from  what  we  have  previously 
observed. 

2.  Of  the  five  programs  that  held,  or  are  planning  to  hold,  a  critical  design  review  during  our 
assessment  period,  two  met  all  of  the  best  practices.  Two  of  the  programs  have  not  fully 
demonstrated  mature  technologies,  but  all  plan  to  release  at  least  90  percent  of  drawings. 
Three  of  5  programs  plan  to  test  a  system-level  integrated  prototype  to  determine  the 
effectiveness  of  their  designs.  This  is  an  improvement  over  our  previous  assessment  when 
none  of  the  programs  implemented  all  of  the  best  practices. 

3.  Of  the  five  programs  that  held  or  plan  to  hold  a  production  decision  during  our  assessment 
period,  one  met  all  of  the  best  practices  including  demonstrating  that  their  manufacturing 
process  capabilities  are  in  control.3  Three  of  the  programs  do  not  plan  to  test  a  production- 
representative  prototype  before  making  a  production  decision,  but  all  state  that  they  have 
demonstrated  processes  on  a  pilot  production  line.  Overall  implementation  is  better  than 
what  we  have  observed  in  the  past. 


Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 

3ln  DOD’s  technical  comments  on  a  draft  of  this  report,  we  were  informed  that  the  MQ-4C  Triton’s 
production  decision  has  been  delayed  to  May  2016. 


1 8 

Because  knowledge  points  and  best  practices  differ  for  shipbuilding  programs,  we 
exclude  the  five  shipbuilding  programs  from  parts  of  our  analysis  at  each  of  the  three 
knowledge  points,  for  more  information  see  appendix  I. 

1 Q 

Information  for  these  programs  was  collected  from  two  data  collection  instruments 
distributed  to  program  officials.  See  the  “Analysis  of  Selected  DOD  Programs  Using 
Knowledge-Based  Criteria”  section  of  appendix  I  for  more  information. 
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1.  The  seven  programs  that  started  system  development,  or  had 
plans  to  do  so  in  2015,  have  knowledge  deficits  which  undermine 
the  foundation  needed  to  avoid  future  cost  and  schedule 
problems.  One  program  plans  to  begin  system  development  with 
immature  technologies  and  none  of  the  seven  programs 
demonstrated  critical  technologies  in  an  operational 
environment.  Five  programs  conducted  all  systems  engineering 
reviews,  including  a  preliminary  design  review,  before  entering 
system  development  while  one  program  conducted  one  of  these 
reviews.  Six  programs  plan  to  limit  their  development  phase  to  6 
years  or  less  whereas  one  has  yet  to  determine  the  length  of  its 
development  phase.  Our  prior  work  shows  that  the  most  critical 
juncture  in  any  major  defense  acquisition  is  the  decision  to  start 
system  development,  a  point  at  which  knowledge-based  acquisition 
practices  recommend  having  a  match  between  what  DOD  wants  in  a 
weapon  system,  as  defined  by  its  requirements,  and  the  mature 
technologies,  funding,  schedule,  and  other  resources  needed  to 
develop  that  system.20  Our  current  assessment  shows  that  the 
implementation  of  best  practices  by  the  7  programs  starting 
development  is  better  from  what  we  have  previously  observed.  Figure 
8  shows  the  extent  to  which  recommended  acquisition  practices  for 
knowledge  point  1  have  been  implemented  for  the  seven  programs 
that  recently  started  system  development  or  plan  to  do  so  in  early 
2016:  Amphibious  Combat  Vehicle  (ACV),  B-2  Defensive 
Management  System  Modernization  (B-2  DMS-M),  Common  Infrared 
Countermeasures  (CIRCM),  Joint  Air-to-Ground  Missile  (JAGM), 
Military  GPS  User  Equipment  Increment  1  (MGUE),  Next  Generation 
Jammer  Increment  1  (NGJ  Inc  1),  and  Offensive  Anti-Surface  Warfare 
Increment  1  (OASuW  Inc  1) — as  well  as  the  other  36  current 
programs  we  assessed  which  accomplished  this  knowledge  point 
previously. 


GAO,  Best  Practices:  Better  Management  of  Technology  Development  Can  Improve 
Weapon  System  Outcomes ,  GAO/NSIAD-99-162  (Washington,  D.C.:  July  30,  1999)  and 
GAO,  Best  Practices:  Better  Matching  of  Needs  and  Resources  Will  Lead  to  Better 
Weapon  System  Outcomes,  GAO-01-288  (Washington,  D.C.:  Mar.  8,  2001). 
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Figure  8:  Implementation  of  Knowledge-Based  Practices  for  Programs  at  System 
Development  Start 
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Demonstrate  Technology 
Maturity 


We  assessed  to  what  level  programs  starting  system  development 
matured  their  critical  technologies.  Federal  statute  requires  that 
programs  generally  obtain  a  certification  stating  that  technologies 
have  been  demonstrated  in  a  relevant  environment.21  However, 
knowledge-based  acquisition  practices  recommend  that  programs 
fully  mature  technologies  by  demonstrating  them  in  an  operational — or 
realistic — environment  prior  to  starting  system  development. 
Demonstrating  technologies  in  an  operational  environment  is  a  better 
indicator  of  whether  a  program  has  achieved  a  resource  and 
requirements  match  as  it  demonstrates  the  technologies’  form,  fit,  and 


21  Demonstration  in  a  relevant  environment  is  Technology  Readiness  Level  (TRL)  6. 
Demonstration  in  an  operational  environment  is  TRL  7.  See  appendix  V  for  detailed 
descriptions  of  TRLs.  In  addition,  a  major  defense  acquisition  program  generally  may  not 
receive  approval  for  development  start  until  the  milestone  decision  authority  certifies  that 
the  technology  in  the  program  has  been  demonstrated  in  a  relevant  environment.  10 
U.S.C.  §  2366b(a)(3)(D).  Under  certain  circumstances  this  requirement  may  be  waived. 
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function  as  well  as  the  effect  of  the  intended  environment  on  those 
technologies. 

One  program  at  development  start — OASuW  Inc  1 — does  not  plan  to 
demonstrate  its  critical  technologies  in  even  a  relevant  environment. 
OASuW  Inc  1  is  a  Navy  program  initiated  in  2013  in  response  to  an 
urgent  operational  need  and  will  be  developed  using  an  accelerated 
acquisition  strategy.  While  the  program  plans  to  leverage  previous 
technology  demonstration  efforts,  it  also  plans  to  begin  development 
in  February  2016  with  immature  technologies.  Currently  the  program 
has  six  critical  technologies  whose  basic  components  have  been 
integrated  and  tested  in  a  lab  environment.  The  components  may  be 
representative  of  a  technically  feasible  approach  but  they  have  not 
been  matured  to  the  point  of  demonstrating  form,  fit,  and  function. 
Beginning  development  with  low  technology  maturity  adds  significant 
risk  to  the  program’s  accelerated  acquisition. 

We  found  that  none  of  the  seven  programs  that  began  or  were 
planning  to  begin  system  development  in  2015  will  have  fully  mature 
technologies.  Fully  mature  technologies  require  that  the  program 
demonstrate  their  technologies  in  an  operational  environment.  For 
example,  the  Navy  assessed  the  ACV’s  critical  technologies  as  fully 
mature  based  on  a  technology  readiness  assessment  approved  in 
December  2014.  This  assessment  was  based  on  the  non- 
developmental  nature  of  the  vehicles,  the  use  of  mature  technology 
for  modifications,  and  tests  and  demonstrations  of  prototype  vehicles 
done  for  another  program,  the  Marine  Personnel  Carrier.  In  contrast, 
we  determined  that  one  of  the  program’s  two  critical  technologies  had 
not  been  tested  in  an  operational  environment,  as  it  was  not  tested  at 
sea,  and  therefore  was  not  fully  mature.22 

Of  the  remaining  36  programs,  3  reported  that  all  of  their  critical 
technologies  were  demonstrated  in  an  operational  environment  when 
they  began  development,  while  23  programs  reported  that  their 
technologies  had  not  reached  full  maturity.  Of  all  of  the  programs  in 
our  assessment,  just  7  percent  began  system  development  with  fully 
mature  technologies. 


22GAO,  Amphibious  Combat  Vehicle:  Some  Acquisition  Activities  Demonstrate  Best 
Practices;  Attainment  of  Amphibious  Capability  to  be  Determined,  GAO-16-22 
(Washington,  D.C.:  October  28,  2015). 
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We  examined  whether  programs  started  before  or  after  2010  are 
meeting  best  practices  and  their  subsequent  levels  of  cost  growth.  Of 
the  36  programs  we  assessed  for  technology  readiness,  20  began 
system  development  before  2010,  15  after,  and  one  did  not  hold  a 
system  development  start.  Eight  of  the  20  programs  began 
development  with  immature  technologies.  Since  they  began  system 
development  they  have  collectively  realized  almost  $32  billion  in 
development  cost  growth.  In  contrast,  the  seven  programs  that  started 
development  with  fully  mature  technologies  before  2010  have  realized 
just  $286  million,  in  development  cost  growth.  The  remaining  5 
programs  did  not  report  on  the  technology  maturity  at  this  point. 


Complete  Systems 
Engineering  Reviews 
before  Development  Start 


•  A  system  functional  review  establishes  a  baseline  for  the  planned 
system. 

•  A  preliminary  design  review  establishes  that  requirements  are 
defined  and  feasible,  and  that  the  proposed  design  can  meet 
those  requirements  within  cost,  schedule,  and  other  system 
constraints. 

We  found  that  six  programs  that  began  or  plan  to  begin  system 
development,  completed  system  functional  and  system  requirements 
reviews  prior  to  starting  system  development  and  one  did  not.  This  is 
an  improvement  over  the  other  36  programs  we  assessed. 

WSARA  established  a  statutory  requirement  to  conduct  a  preliminary 
design  review  prior  to  entering  system  development.23  Among  the 
seven  programs  that  started  system  development  in  2015,  or  planned 


Knowledge-based  acquisition  practices  recommend  that  programs 
hold  systems  engineering  events  before  the  start  of  system 
development: 

•  A  system  requirements  review  ensures  that  requirements  have 
been  properly  identified  and  that  there  is  a  mutual  understanding 
between  the  government  and  the  contractor. 


oo 

Pub.  L.  No.  1 1 1-23,  §  205(a).  A  major  defense  acquisition  program  may  not  receive 
milestone  B  approval  until  the  program  has  held  a  preliminary  design  review  and  the 
milestone  decision  authority  has  conducted  a  formal  post-preliminary  design  review 
assessment  and  certified  on  the  basis  of  such  assessment  that  the  program  demonstrates 
a  high  likelihood  of  accomplishing  its  intended  mission  unless  a  waiver  is  properly  granted 
by  the  milestone  decision  authority.  10  U.S.C.  §§  2366b(a)(2),  (d)(1 ). 


Page  35 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


to  in  early  2016,  five  of  them  conducted  a  preliminary  design  review 
before  beginning  system  development.  The  other  two  programs 
received  a  waiver  to  this  requirement  and  expect  to  conduct  this 
review  after  their  system  development  start.  Thirteen  of  the  other  36 
programs  held  a  preliminary  design  review  before  the  start  of  system 
development.  Nearly  60  percent  of  programs  that  began  development 
since  2010  are  implementing  this  practice  compared  to  25  percent  of 
programs  that  began  development  before  2010,  an  improvement  over 
our  prior  assessment. 


Constrain  System 
Development  Phase  to  6 
years  or  Less 


Knowledge-based  acquisition  practices  also  recommend  that  a 
program  constrain  the  system  development  phase  to  6  years  or  less. 
Our  review  of  the  seven  programs  that  began,  or  plan  to  begin, 
system  development  during  our  review  period  found  that  six  of  the 
seven  currently  plan  to  do  so.  One  program  in  our  assessment  has 
not  determined  when  it  will  end  system  development  and  enter 
production. 

For  the  remaining  36  programs  we  assessed,  22  planned  to  limit  their 
system  development  phase  to  six  years  or  less  at  the  time  they 
started  system  development.24  Plans  to  constrain  the  development 
phase  at  the  start  of  system  development  are  not  always  successful. 
For  example,  5  of  the  programs  we  assessed  took  longer  to  reach 
production  than  the  six  years  or  less  they  originally  planned.  One  of 
these  programs  began  development  with  immature  technologies  and 
4  had  technologies  approaching  maturity.  As  a  group,  they  reported 
approximately  $2.1  billion  in  development  cost  growth  from  first  full 
estimates.  Sixteen  programs  plan  to  enter  production  in  the  future,  but 
additional  delays  may  occur,  especially  as  a  number  of  these 
programs  have  not  implemented  all  of  the  best  practices  for  system 
development. 


24We  did  not  assess  shipbuilding  programs  against  this  recommended  practice  to  limit  the 
development  phase,  as  their  development  cycles  do  not  align  in  a  manner  consistent  with 
other  programs. 
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As  part  of  our  analysis,  we  also  assessed  12  programs  scheduled  to 
start  system  development  and  become  major  defense  acquisition 
programs  in  the  near  future.  The  greatest  point  of  leverage  to  ensure 
a  program’s  success  is  at  the  start  of  a  new  program.  At  this  point, 
decision  makers  can  ensure  that  new  programs  implement  best 
practices  and  exhibit  desirable  principles  before  they  are  approved 
and  funded.  These  programs  provided  information  on  the  knowledge 
they  planned  to  obtain  and  the  best  practices  they  intend  to  implement 
before  system  development  start.  Five  of  the  12  identified  critical 
technologies  and  their  anticipated  maturity  levels  expected  at  system 
development  start.  Two  programs — the  Unmanned  Carrier-Launched 
Airborne  Surveillance  and  Strike  system  and  the  Fleet  Replenishment 
Oiler — reported  that  they  expect  their  critical  technologies  to  be  fully 
mature.  Table  6  shows  this  and  that  the  remaining  programs  reported 
that  their  critical  technologies  would  not  be  fully  mature  or  have  not 
been  identified  and  one  program’s  technology  maturity  is  to  be 
determined  at  the  time  of  their  system  development  start. 


Table  6:  Projected  Implementation  of  Knowledge-Based  Practices  for  Future  Programs 


Development 

start 
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demonstrate  all 
critical  technologies 
in  an  operational 
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Begin  Future  Programs 
with  Adequate  Knowledge 
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Three  of  the  12  programs  plan  to  conduct  all  of  the  recommended 
systems  engineering  reviews  before  development  start,  including  a 
system  functional  review,  a  system  requirements  review,  and  a 
preliminary  design  review.  Unlike  the  programs  that  held  system 
development  start  in  the  past  year,  only  four  of  the  1 2  future  programs 
plan  to  hold  a  preliminary  design  review  before  the  start  of  system 
development.  While  five  of  the  12  future  programs  currently  plan  to 
limit  their  system  development  phase  to  6  years  or  less,  these  plans 
are  preliminary  and  the  programs  are  at  risk  of  not  satisfying  all  the 
knowledge-based  practices  we  reviewed,  leaving  them  at  risk  for  cost 
and  schedule  growth. 

If  DOD  decides  to  let  these  programs  proceed  without  the  knowledge 
required  to  achieve  a  requirements  and  resource  match  this  will  have 
larger  implications  than  the  expected  outcomes  on  these  programs.  It 
sends  a  signal  across  the  entire  portfolio  of  current  and  future 
programs  about  what  is  acceptable  in  terms  of  following  a  knowledge- 
based  acquisition  approach.  It  is  imperative  that  top  decision  makers 
ensure  that  new  programs  exhibit  desirable  principles  that  embody 
knowledge-based  acquisition  best  practices  before  they  are  approved 
and  funded  at  the  start  of  system  development,  one  of  the  key  points 
in  the  acquisition  cycle  where  discipline  and  accountability  can  be 
established  and  reinforced. 


2.  Of  the  five  programs  that  held,  or  are  planning  to  hold,  a  critical 
design  review  during  our  assessment  period,  two  met  all  of  the 
best  practices.  Two  of  the  programs  have  not  demonstrated 
mature  technologies,  and  all  plan  to  release  at  least  90  percent  of 
drawings.  Three  of  5  programs  plan  to  test  a  system-level 
integrated  prototype.  For  this  knowledge  point  in  a  program’s 
acquisition,  we  assessed  eight  best  practices  to  determine  the  extent 
to  which  programs  are  attaining  the  knowledge  needed  for  a  low  risk 
acquisition.  We  found  an  improvement  over  our  previous  assessment 
in  programs’  implementation  of  best  practices  at  their  critical  design 
review.  We  have  previously  reported  that  programs  that  hold  their 
critical  design  review  before  achieving  knowledge  of  a  stable, 
demonstrated  design  also  experience  higher  average  costs  and 
longer  schedule  delays.  Figure  9  below  outlines  the  implementation  of 
this,  as  well  as  other,  best  practices  among  the  programs  we 
assessed. 
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Figure  9:  Implementation  of  Knowledge-Based  Practices  for  Programs  at  Critical 
Design  Review 
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Demonstrate  Technology 
Maturity 


As  product  knowledge  is  cumulative,  by  the  critical  design  review 
programs  should  have  demonstrated  critical  technologies  in  an 
operational  environment  to  ensure  that  the  product  can  meet 
requirements.  Failure  to  fully  mature  technologies  prior  to  developing 
the  system  design  can  lead  to  redesign  and  cost  and  schedule  growth 
if  later  discoveries  in  maturing  technologies  cause  revisions. 

Two  programs  do  not  plan  to  meet  this  best  practice  before 
completing  their  critical  design  review — ACV  and  AMDR.  At  the  time 
of  our  review,  the  ACV  program  was  scheduled  to  hold  their  critical 
design  review  in  February,  but  this  may  be  delayed  due  to  a  bid 
protest  related  to  the  development  contract  awards.  The  ACV 
program’s  current  level  of  technology  maturity  is  discussed  in  the 
previous  section.  AMDR  has  four  critical  technologies  that  have  been 
demonstrated  in  a  relevant,  but  not  an  operational  environment,  at  the 
time  of  its  critical  design  review. 


Page  39 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


To  assess  the  effect  of  technology  maturity  on  cost  over  time,  we 
examined  the  22  programs  that  started  development  5  or  more  years 
ago  and  the  level  of  their  technology  maturity  at  the  time  of  their 
critical  design  review.  Of  these  22  programs,  seven  held  their  critical 
design  review  with  immature  technologies.  These  programs  have 
realized  an  average  of  74  percent,  or  $30  billion,  in  development  cost 
growth  over  their  initial  cost  estimates.  Five  of  these  programs  are 
among  those  mentioned  in  the  previous  section  that  started  system 
development  with  immature  technologies  with  the  addition  of  the 
M109A7  Family  of  Vehicles  and  Family  of  Advanced  Beyond  Line-of- 
Sight  Terminals  (FAB-T).  In  contrast,  9  programs  had  technologies 
approaching  maturity  or  fully  mature  at  their  critical  design  review. 
These  programs  have  realized  an  average  of  33  percent,  or  $3.2 
billion,  in  development  cost  growth,  significantly  less  than  the 
programs  with  immature  technologies. 


Demonstrate  System 
Design  Stability 


Of  the  five  programs  that  held,  or  plan  to  hold,  their  critical  design 
review  in  2015,  all  demonstrated  or  are  expected  to  demonstrate 
design  stability  by  releasing  over  90  percent  of  their  expected  design 
drawings.  This  is  an  improvement  over  our  prior  assessment,  in  which 
both  programs  we  assessed  had  not  met  this  practice.  When 
compared  to  the  other  26  programs  we  assessed  that  held  critical 
design  reviews  previously;  6  met  this  best  practice,  1 1  had  not,  and  9 
reported  that  the  practice  was  not  applicable. 

Testing  of  a  system-level  integrated  prototype  is  useful  for 
demonstrating  that  a  system  will  work  as  intended  and  can  be  built 
within  cost  and  schedule.  Two  of  the  five  programs  that  held,  or  plan 
to  hold,  their  critical  design  review  in  2015  did  not,  or  will  not,  test  a 
system-level  integrated  prototype  prior  to  their  design  review — ACV 
and  Space  Fence,  Inc  1 .  The  ACV  program  reported  completing  this 
practice  on  another  vehicle  prototype.  This  vehicle  was  not  the  same 
design  as  the  current  ACV.  As  a  result,  GAO  determined  that  this  test 
did  not  meet  the  best  practice  as  it  did  not  include  all  of  the  key 
subsystems  and  components  that  will  be  in  the  ACV.  The  Space 
Fence  Inc  1  program  is  developing  a  large,  ground-based  radar 
intended  to  track  objects  in  Earth’s  orbit.  The  program  conducted  its 
design  review  in  June  2015  and  completed  at  least  90  percent  of  its 
design  drawings,  but  did  not  demonstrate  its  ability  to  meet 
performance  requirements  with  testing  of  a  system-level  integrated 
prototype.  The  program  plans  to  verify  the  design  through  testing  on  a 
fully  configured  prototype,  with  some  production-representative 
elements,  beginning  in  2016  after  the  start  of  production. 
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We  assessed  26  other  programs  that  held  a  critical  design  review 
prior  to  2015,  and  found  that  only  one  tested  an  early  integrated 
prototype  before  its  critical  design  review.  For  the  other  programs, 
early  integrated  prototype  testing  occurred  or  will  occur  after  their 
critical  design  reviews.  In  some  cases  this  testing  will  occur  years 
later.  We  did  not  assess  shipbuilding  programs  against  this 
knowledge-based  acquisition  practice  as  testing  early  system 
prototypes  in  these  programs  may  not  be  practical  as  these  programs 
are  not  delivering  large  quantities.  The  limited  use  of  this  testing 
before  design  review  among  the  programs  we  assessed  shows  no 
improvement  from  our  prior  assessments. 

The  five  programs  we  assessed  all  plan  to  implement  the  last  five 
knowledge-based  practices  shown  in  figure  9  to  increase  confidence 
in  the  stability  of  their  product’s  design  and  its  effect  on  production. 
This  includes  practices  such  as  establishing  a  reliability  growth  curve 
and  identifying  key  product  characteristics  in  preparation  for 
production  activities.  For  the  other  26  programs  in  this  assessment 
that  have  already  entered  production,  a  majority  of  them  reported 
using  each  of  these  practices. 


Prepare  for  Production  3.  Of  the  five  programs  that  held  a  production  decision  during  our 

assessment  period,  one  met  all  of  the  best  practices,  including 
demonstrating  that  their  manufacturing  process  capabilities  are 
in  control.  Three  of  the  programs  do  not  plan  to  test  a 
production-representative  prototype  before  making  a  production 
decision.  All  five  state  that  they  have  demonstrated 
manufacturing  processes  on  a  pilot  production  line.  Capturing 
critical  manufacturing  knowledge  before  entering  production  ensures 
that  a  weapon  system  will  work  as  intended  and  can  be  manufactured 
efficiently  to  meet  cost,  schedule,  and  quality  targets.  This  knowledge 
can  be  captured  through  the  use  of  various  proactive  methods, 
including  the  use  of  statistical  process  control  data,  pilot  production 
lines,  manufacturing  readiness  levels,  and  prototype  testing.  Overall 
we  found  better  implementation  of  best  practices  at  this  point  than 
what  we  have  observed  in  the  past.  Figure  10  shows  the  extent  to 
which  programs  that  have  held  a  production  decision  have 
implemented  associated  knowledge-based  practices. 
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Figure  10:  Implementation  of  Knowledge-Based  Practices  for  Programs  at 
Production  Decision 


Knowledge-based  practices 
at  production  start 


For  the  11 
non-shipbuilding 
programs  that 
have  reached 
this  juncture11 


Demonstrate  all  critical  technologies  in  an 
operational  environment 
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Release  at  least  90  percent  of  drawings 
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Demonstrate  manufacturing  process 
capabilities  are  in  control 
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Demonstrate  critical  processes  on  a  pilot 
production  line 
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• 
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Test  a  production-representative  prototype 
in  its  intended  environment 

o 

• 

o 

• 

o 

4 

7 

0 

0  Practice  implemented 

O  Practice  not  implemented 

— —  Practice  not  applicable  or  information  not  available  per  the  program  office  response 

FAB-T  -  Family  of  Advanced  Beyond  Line-of-Sight  Terminals 

JLTV  -  Joint  Light  Tactical  Vehicle 

MQ-4C  -  MQ-4C  Triton  Unmanned  Aircraft  System 

SDB  II  -  Small  Diameter  Bomb  Increment  II 

SSC  -  Ship  to  Shore  Connector  Amphibious  Craft 

Source:  GAO  analysis  of  DOD  data.  |  GAO-16-329SP 


aln  DOD’s  technical  comments  on  a  draft  of  this  report,  we  were  informed  that  the  MQ-4C  Triton’s 
production  decision  has  been  delayed  to  May  2016. 

bShipbuilding  programs  are  not  included  in  the  assessment  of  best  practices  at  production  decision. 


Demonstrate 
Manufacturing  Processes 
Are  in  Control 


Four  of  the  five  programs  that  have  held  recent  production  decisions 
did  not  demonstrate  that  their  manufacturing  process  capabilities  were 
in  control.  Small  Diameter  Bomb  Increment  II  was  the  only  program 
that  satisfied  this  best  practice.  The  programs  not  meeting  this 
practice  are  tracking  manufacturing  readiness  levels,  although  none  of 
the  programs  reached  the  best  practices  recommended  level  of 
readiness.  Our  prior  work  has  shown  that  capturing  critical 
manufacturing  knowledge  before  entering  production  helps  ensure 
that  the  system  will  work  as  intended  and  will  meet  cost,  schedule  and 
quality  targets.  Of  the  remaining  11  non-ship  programs  we  assessed 
that  held  production  decisions  prior  to  2015,  1  provided  data  indicating 
that  critical  manufacturing  processes  were  in  control  at  the  time  of 
their  production  start. 
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Demonstrate  Critical 
Processes  on  a  Pilot 
Production  Line 


All  of  the  programs  that  held  a  production  decision  over  the  past  year 
demonstrated  critical  processes  on  a  pilot  production  line.  Of  the 
programs  we  assessed,  17  plan  to  hold  a  production  decision  in  the 
future  and  they  all  indicated  that  they  intended  to  test  a  pilot 
production  line  before  production  start.  Demonstrating  critical 
processes  on  a  pilot  production  line  is  an  important  initial  step  to 
ensure  that  all  critical  manufacturing  processes  are  repeatable, 
sustainable,  and  capable  of  consistently  producing  parts  within  the 
product’s  quality  tolerances  and  standards — at  the  start  of  production. 
Of  the  1 1  non-ship  programs  that  held  productions  decisions  in  the 
past,  only  six  have  demonstrated  this  best  practice  whereas  two  did 
not  and  three  programs  replied  that  the  practice  was  not  applicable. 


Test  a  Production- 
Representative  Prototype 
in  Its  Intended 
Environment 


Our  body  of  work  on  a  knowledge-based  approach  shows  that 
production  and  postproduction  costs  are  also  minimized  when  a  fully 
integrated,  production-representative  prototype  is  demonstrated  prior 
to  the  production  decision  as  making  design  changes  after  production 
begins  can  be  both  costly  and  inefficient.  Of  the  5  programs  that 
recently  accomplished  this  milestone,  2  plan  to  meet  this  best  practice 
and  3  do  not.  The  three  programs — FAB-T,  MQ-4C,  and  SSC — plan 
to  complete  this  testing  from  1 0  months  to  over  a  year  after  their 
production  decision  dates.  Of  the  17  programs  that  plan  to  hold  their 
production  decision  in  the  future,  9  report  that  they  intend  to  test  a 
fully  configured  prototype.  In  not  conducting  testing  of  production- 
representative  prototypes  prior  to  beginning  manufacturing,  programs 
risk  discovering  issues  late  in  testing,  which  may  trigger  the  need  for 
expensive  re-tooling  of  production  lines  and  retrofitting  of  articles  that 
have  completed  production.  Of  the  11  non-ship  programs  with 
production  decisions  in  the  past,  four  reported  testing  a  production- 
representative  prototype  before  this  decision  whereas  seven  did  not. 
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Observations  about 
DOD’s 

Implementation  of 
Key  Acquisition 
Reform  Initiatives  and 
Program  Concurrency 


Over  the  past  5  years,  increasing  numbers  of  DOD  programs  have  been 
implementing  acquisition  reform  initiatives  such  as  those  in  WSARA,  and 
the  “Better  Buying  Power”  initiatives — many  of  which  have  been 
incorporated  into  the  latest  iteration  of  DOD’s  Instruction  5000. 02. 25  The 
implementation  of  these  initiatives  has  led  to  better  acquisition  outcomes 
on  some  programs.  Nevertheless,  DOD  still  faces  challenges  in  fully 
implementing  these  reforms.  Given  the  level  of  DOD’s  investment  in 
major  defense  acquisition  programs  continued  attention  is  warranted. 

We  focused  our  analysis  on  the  aspects  of  WSARA  and  DOD’s  “Better 
Buying  Power”  initiatives  that  address  program  and  portfolio  affordability; 
cost  growth  controls;  and  the  use  of  competition  throughout  the 
acquisition  life-cycle.  In  addition,  we  reviewed  programs’  software 
development  efforts  to  determine  how  they  monitor  and  manage  these 
efforts.  We  also  assessed  the  amount  of  planned  concurrency  between 
developmental  testing  and  production  for  current  programs  in  the 
portfolio. 


Overall,  we  found  modest  improvements  across  many  of  the  areas  we 
assessed.  Compared  to  our  previous  assessment,  similar  proportions  of 
programs  have  established  affordability  constraints,  and  more  programs 
have  attempted  to  determine  and  realize  “should-cost”  savings.  However, 
the  number  of  current  and  future  programs  using  or  planning  to  use 
competition  measures  does  not  show  substantial  improvement.  Other 
risks  to  program  performance  persist  as  well,  as  high  levels  of 
concurrency  between  development  testing  and  production  may  add  risk  to 
the  portfolio. 


Our  analysis  allows  us  to  make  the  following  five  observations  concerning 
key  acquisition  reform  initiatives  and  program  concurrency: 


25Department  of  Defense  Instruction  5000.02,  Operation  of  the  Defense  Acquisition 
System  (Jan.  2015)  (“DOD  Instruction  5000.02”). 
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Acquisition  Reform  and  Concurrency  Observations 

1.  Of  the  55  current  and  future  programs  we  assessed,  37  have  established  an  affordability 
constraint,  similar  to  the  implementation  rate  in  our  last  assessment.  The  development  cost 
growth  for  the  current  programs  we  assessed  with  an  affordability  constraint  is  33 
percentage  points  lower  than  that  for  programs  without  a  constraint. 

2.  Of  the  43  current  programs  we  assessed,  39  have  conducted  a  “should-cost”  analysis 
resulting  in  anticipated  savings  of  over  $35  billion;  approximately  $21  billion  of  these  savings 
have  been  realized  to  date. 

3.  Of  the  55  current  and  future  programs  we  assessed,  43  plan  to  promote  competition  at 
some  point  during  acquisition.  Eight  of  these  programs  have  no  plans  for  competition  before 
or  after  development  start  and  only  half  of  the  future  programs  we  assessed  plan  to  conduct 
competitive  prototyping. 

4.  Of  the  55  current  and  future  programs  we  assessed,  40  reported  software  development  as  a 
high-risk  area.  Programs  that  did  not  report  their  software  development  as  high-risk  have 
experienced  greater  schedule  delays.  Sixteen  of  the  43  current  programs  we  assessed  plan 
to  concurrently  conduct  production  and  software  development. 

5.  Sixteen  of  the  43  current  programs  we  assessed  are  in  production.  Eleven  of  these 
programs  plan  to  complete  30  percent  or  more  of  their  developmental  testing  concurrent 
with  production.  Further,  three  of  these  1 1  programs  plan  to  place  more  than  20  percent  of 
its  procurement  quantities  under  contract  before  testing  is  complete.  For  the  programs  we 
assessed,  as  concurrency  increases,  so  does  total  acquisition  cost  growth. 


Additional  details  about  each  observation  follow. 

1.  Of  the  55  current  and  future  programs  we  assessed,  37  have 
established  an  affordability  constraint,  similar  to  the 
implementation  rate  in  our  last  assessment.  The  development 
cost  growth  for  programs  we  assessed  with  an  affordability 
constraint  is  33  percentage  points  lower  than  that  for  programs 
without  a  constraint.  In  2010,  DOD  launched  a  series  of  “Better 
Buying  Power”  initiatives  with  the  goal  of  delivering  superior 
capabilities  to  the  warfighter  and  better  value  to  the  taxpayer.  One 
such  initiative  was  the  establishment  of  an  affordability  analysis  that 
results  in  cost  constraints.  This  analysis  differs  from  program  cost 
estimates  in  that  the  constraint  serves  as  a  program  requirement  to 
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ensure  that  the  program  remains  cost-effective.26  In  accordance  with 
DOD  Instruction  5000.02,  affordability  constraints  are  intended  to 
force  prioritization  of  requirements,  enable  cost  trades,  and  ensure 
that  unaffordable  programs  do  not  enter  the  acquisition  process. 

When  approved  affordability  constraints  cannot  be  met,  a  program’s 
technical  requirements,  schedule,  and  required  quantities  must  be 
revisited.  Failure  to  remain  within  these  constraints  may  result  in 
program  termination. 

Of  the  current  programs  we  assessed,  30  of  43  have  established  an 
affordability  constraint.  A  similar  proportion  of  programs  had 
implemented  affordability  constraints  in  our  last  assessment.  Two 
programs,  the  B-2  Defensive  Management  System  Modernization  and 
the  Next  Generation  Operational  Control  System,  responded  that  they 
do  not  currently  expect  to  meet  their  affordability  constraints.  Of  the 
remaining  13  programs  that  have  not  established  an  affordability 
constraint,  most  began  system  development  before  this  requirement 
was  established.27  Similarly,  of  the  12  future  programs  we  assessed, 
seven  reported  that  they  established  an  affordability  constraint. 

The  effectiveness  of  these  constraints  has  yet  to  be  widely  tested; 
however,  the  programs  we  assessed  that  had  established  an 
affordability  constraint  experienced  a  median  of  4  percent  growth  in 
development  costs  from  their  original  baselines.  By  contrast,  the 
programs  we  assessed  without  an  affordability  constraint  experienced 
a  median  of  37  percent  growth  in  development  costs.28 

2.  Of  the  43  current  programs  we  assessed,  39  have  conducted  a 
“should-cost”  analysis  resulting  in  anticipated  savings  of  over 
$35  billion;  approximately  $21  billion  of  these  savings  have  been 
realized  to  date.  DOD’s  “Better  Buying  Power”  initiatives  also 


26Affordability  goals  are  established  at  milestone  A,  the  entry  into  technology 
development.  After  systems  engineering  trade-offs  are  completed  during  the  technology 
development  phase,  these  affordability  goals  then  become  affordability  caps  prior  to 
milestone  B,  the  start  of  system  development,  when  a  match  is  to  be  made  between 
requirements  and  resources.  We  refer  to  the  goals  and  caps  collectively  as  affordability 
constraints. 

27The  Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  program  has  a  congressionally 
mandated  cost  cap,  which  we  do  not  consider  the  same  as  the  affordability  requirement 
considered  in  this  analysis.  As  a  result  we  include  this  program  in  the  total  number  of 
programs  without  an  affordability  cap. 

oo 

For  this  analysis  we  took  the  median  cost  growth  associated  with  each  group  of 
programs  to  mitigate  the  effects  of  programs  with  significant  cost  changes. 
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emphasize  the  importance  of  driving  cost  improvements  during 
contract  negotiation  and  program  execution  to  control  costs  both  in 
the  short-term  and  throughout  the  product  life  cycle.  In  accordance 
with  DOD  Instruction  5000.02,  each  program  must  conduct  a  “should- 
cost”  analysis  resulting  in  an  estimate  to  be  used  as  a  management 
tool  to  control  and  reduce  cost.  “Should-cost”  analysis  can  be  used  to 
justify  each  cost  under  the  program’s  control  with  the  aim  of  reducing 
negotiated  prices  for  contracts  and  obtaining  other  efficiencies  in 
program  execution  to  bring  costs  below  those  budgeted  for  the 
program.  Any  savings  achieved  can  then  be  reallocated  within  the 
program  or  for  other  priorities. 

We  found  that  39  out  of  43  current  programs  conducted  a  “should- 
cost”  analysis.  Three  of  the  four  current  programs  that  reported  not 
conducting  a  “should-cost”  analysis  are  in  the  process  of  completing 
one.  Of  the  39  programs,  35  identified  approximately  $35  billion  in 
realized  and  future  savings.  Those  programs  cited  several  activities 
as  responsible  for  some  or  all  of  their  “should-cost”  savings,  including 

•  efficiencies  realized  through  contract  negotiations  (15  programs), 

•  design  trades  to  balance  affordability  and  capability  (12 
programs),  and 

•  developmental  or  operational  testing  efficiencies  (7  programs). 

Twenty-six  of  the  35  programs  reported  $21.2  billion  in  realized 
“should-cost”  savings.  Of  this  amount  $3.4  billion  in  savings  accrued 
from  savings  in  development  costs,  $17.6  billion  from  procurement 
costs,  and  $0.2  billion  from  other  sources.  Programs  also  provided 
insight  as  to  how  their  realized  savings  were  allocated.  Figure  1 1 
shows  the  amount  of  savings  reallocated  to  other  purposes. 
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Figure  11:  Priorities  for  the  $21.2  Billion  in  Total  Realized  “Should-Cost”  Savings 

Realized  “should-cost”  savings 

(in  fiscal  year  2016  dollars  billion) 


Went  to  outside  service  priorities 
Went  to  service  priorities 
Kept  in  program 
Destination  unknown 


Source:  GAO  analysis  of  survey  data.  |  GAO-16-329SP 


Of  the  $21.2  billion  in  realized  “should-cost”  savings,  $6  billion  was 
kept  within  the  programs  to  fund  other  priorities.  Programs  reported 
that  nearly  $286  million  of  those  savings  were  used  to  offset  budget 
cuts  required  by  sequestration.  Another  $5.6  billion  of  the  realized 
“should-cost”  savings  went  to  priorities  within  the  service  to  which  the 
program  belongs  and  $2.4  billion  went  to  priorities  outside  of  the 
service.  While  delivering  value  to  the  taxpayer  in  the  acquisition 
process  is  one  of  DOD’s  stated  objectives,  programs  may  not  have 
strong  incentives  to  realize  or  report  “should-cost”  savings  if  those 
programs  perceive  them  as  resulting  in  the  funding  of  other  DOD 
priorities.  Programs  we  assessed  were  unable  to  account  for  the 
destination  of  roughly  a  third  of  the  savings  they  reported. 

Current  programs  that  we  surveyed  expect  to  realize  another  $13.8 
billion  in  future  should  cost  savings.  Of  this  amount,  $8.5  billion  in 
savings  is  expected  to  accrue  for  development  and  procurement 
“should-cost”  savings  and  $5.3  billion  is  expected  from  other  sources. 
Of  the  12  future  programs  we  assessed,  3  reported  having  conducted 
a  “should-cost”  analysis,  and  identified  $9.6  billion  in  future 
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development  and  procurement  savings  and  an  additional  $7.9  billion 
in  other  savings. 

3.  Of  the  55  current  and  future  programs  we  assessed,  43  plan  to 
promote  competition  at  some  point  during  the  acquisition  cycle. 
Eight  of  these  programs  have  no  plans  for  competition  before  or 
after  development  start  and  only  half  of  the  future  programs  we 
assessed  plan  to  conduct  competitive  prototyping.  Competition  is 
a  critical  tool  for  achieving  the  best  return  on  the  government’s 
investment.  Major  defense  acquisition  programs  are  generally 
required  to  plan  for  the  use  of  prototypes  from  two  or  more  contractors 
before  a  program  starts  system  development  and  have  acquisition 
strategies  that  ensure  the  option  of  continued  competition  throughout 
the  acquisition  life  cycle.29  According  to  DOD,  the  fostering  of 
competitive  environments  is  a  central  tenet  in  acquisition  reform  and 
the  single  best  way  to  motivate  contractors  to  provide  the  best  value. 

Figure  12  shows  how  current  programs  plan  to  promote  competition: 


29Pub.  L.  No.  111-23,  §§  202,  203,  as  implemented  in  DOD  Instruction  5000.02.  Under 
certain  circumstances  this  requirement  may  be  waived. 
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Figure  12:  Current  Programs’  Plans  to  Promote  Competition 

Competition  before  system  development  start 


21 


Competition  after  system  development  start 


16 


26 


Competition  before  and  after  system  development  start 


Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 


Of  the  43  current  programs  we  assessed,  21  conducted  or  planned  to 
conduct  competitive  prototyping  before  development  start  and  26 
have  acquisition  strategies  that  include  some  measure  to  encourage 
competition  after  development  start,  such  as  funding  of  next 
generation  prototype  systems  or  subsystems.  Thirteen  current 
programs  reported  pursuing  measures  to  promote  competition  both 
before  and  after  the  start  of  system  development.  These  programs 
experienced  less  development  cost  growth  than  those  that  promoted 
competition  in  only  one  phase  of  acquisition.  Eight  programs  reported 
no  plans  to  pursue  competition  either  before  or  after  beginning  system 
development  and  one  could  not  describe  its  current  strategy  for 
competition.  Six  of  these  are  programs  new  to  the  portfolio  within  the 
last  two  years. 

Figure  13  shows  how  future  programs  plan  to  promote  competition: 
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Figure  13:  Future  Programs’  Plans  to  Promote  Competition 


Competition  before  system  development  start 


No 

Yes 

Before  only 
After  only 
Both 

Don't  know 


Source:  GAO  analysis  of  DOD  data.  |  GAO-16-329SP 


Six  of  12  future  programs  reported  plans  to  conduct  competitive 
prototyping  before  the  start  of  system  development;  the  other  six 
indicated  they  would  not.  Our  prior  work  has  shown  that  competitive 
prototyping  can  help  programs  reduce  technical  risk,  refine 
requirements,  and  validate  designs  and  cost  estimates  prior  to  making 
major  commitments  of  resources.30  Programs  not  taking  this  step  may 
be  missing  an  opportunity  to  lower  costs  and  reduce  risk.  Additionally, 
five  future  programs  indicated  that  they  would  pursue  measures  to 
promote  competition  after  the  start  of  system  development.  Five  of  the 
future  programs  were  unable  to  provide  us  with  information  on  their 
plans  after  development  start. 


on 

GAO,  National  Defense:  Department  of  Defense’s  Waiver  of  Competitive  Prototyping 
Requirement  for  the  Navy’s  Fleet  Replenishment  Oiler  Program,  GAO-1 5-57R 
(Washington,  D.C.:  Oct.  8,  2014). 
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We  asked  both  current  and  future  programs  about  specific  measures 
they  intend  to  pursue  in  order  to  promote  competition  after  the  start  of 
system  development.  The  programs  reported  most  frequently  that 
they  have  or  intend  to 

•  hold  periodic  system  or  program  reviews  to  address  long-term 
competitive  effects  of  program  decisions, 

•  use  modular,  open  architectures  to  enable  competition  for 
upgrades;  and 

•  acquire  complete  technical  data  packages. 

This  is  significant,  as  we  have  previously  found  that  use  of  these 
strategies  can  reduce  product  development  time  and  life-cycle  costs, 
increase  competition  and  innovation,  and  enable  interoperability 
between  systems.31 

4.  Of  the  55  current  and  future  programs  we  assessed,  40  reported 
software  development  as  a  high-risk  area.  Programs  that  did  not 
report  their  software  development  as  high-risk  have  experienced 
greater  schedule  delays.  Sixteen  of  the  43  current  programs  we 
assessed  plan  to  concurrently  conduct  production  while 
completing  software  development.  We  found  in  2004  that  major 
defense  acquisition  programs  were  becoming  increasingly  reliant  on 
software  to  achieve  their  performance  characteristics.32  Software 
development  has  similar  phases  to  that  of  hardware  and — in  the  case 
of  new  systems — occurs  in  parallel  with  hardware  development  until 
software  and  hardware  components  are  integrated. 

Of  the  43  current  programs  we  surveyed,  33  indicated  that  they  had 
identified  software  development  as  a  high-risk  area.  Of  the  12  future 
programs,  seven  identified  their  software  development  as  high  risk. 
This  is  a  larger  proportion  than  what  we  observed  in  our  last 
assessment,  where  25  of  38  current  programs  identified  software 
development  as  high-risk.  Programs  often  identified  more  than  one 


31GAO,  Defense  Acquisitions:  Review  of  Private  Industry  and  Department  of  Defense 
Open  Systems  Experiences,  GAO-14-617R  (Washington,  D.C.:  June  26,  2014). 

oo 

GAO,  Defense  Acquisitions:  Stronger  Management  Practices  Are  Needed  to  Improve 
DOD’s  Software-Intensive  Weapon  Acquisitions,  GAO-04-393  (Washington,  D.C.:  March 
1,2004). 
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reason  for  designating  software  as  high  risk.  The  three  most  common 
reasons  included 

•  27  programs  that  cited  the  challenge  of  completing  the  software 
development  needed  to  conduct  developmental  testing; 

•  26  programs  underestimated  the  difficultly  of  their  originally 
planned  software  effort;  and 

•  17  programs  required  hardware  design  changes,  which 
necessitated  additional  software  development. 

A  similar  proportion  of  programs  in  our  last  assessment  identified 
these  issues  as  motivation  for  identifying  software  as  a  high-risk  area, 
which  may  indicate  that  these  areas  are  due  additional  attention  from 
program  managers.  Ten  programs  reported  that  they  had  not 
identified  software  as  a  risk,  including  one  program  that  expects  to 
begin  production  before  completing  software  development.  Those 
programs  experienced,  on  average,  nine  additional  months  of 
schedule  delays  compared  to  programs  that  had  identified  software  as 
a  high-risk  area.  In  a  few  cases,  those  delays  are  directly  attributable 
to  software-development  related  challenges.  The  challenges 
associated  with  software  development  have  featured  in  a  number  of 
saliently  poor  acquisition  outcomes  in  DOD,  consequently  it  is 
encouraging  that  more  programs  have  begun  to  identify  it  as  a  risk 
area. 

Sixteen  of  the  43  current  programs  we  assessed  plan  to  begin 
production  prior  to  completing  the  software  development  necessary 
for  integration  with  system  hardware  and  achieving  baseline 
capabilities,  a  smaller  proportion  than  our  previous  assessment.  DOD 
policy  allows  for  some  degree  of  concurrency  between  initial 
production  and  the  completion  of  developmental  testing,  especially  for 
the  completion  of  software.  While  some  concurrency  may  be 
necessary  when  rapidly  fielding  urgently  needed  capabilities,  pursuing 
software  development  while  the  system  is  in  production  may  introduce 
risks  if  problems  are  discovered  late  in  testing.  For  example,  we  have 
reported  on  the  F-35  Joint  Strike  Fighter  program,  whose  problems 
necessitated  hardware  changes  to  supplement  the  software  or 
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required  the  acceptance  of  software  whose  reliability  falls  short  of 
overall  system  requirements.33 

5.  Sixteen  of  the  43  current  programs  we  assessed  are  in 
production.  Eleven  of  these  programs  plan  to  complete  30 
percent  or  more  of  their  developmental  testing  concurrent  with 
production.  Further,  3  of  these  11  programs  plan  to  place  20 
percent  or  more  of  their  procurement  quantities  under  contract 
before  testing  is  complete.  For  the  programs  we  assessed,  as 
concurrency  increases,  so  does  total  acquisition  cost  growth. 
Development  testing  is  intended  to  demonstrate  that  a  chosen  design 
has  the  capabilities  required  and  to  discover  and  fix  problems  before  a 
system  enters  production.  Though  DOD  policy  allows  some  degree  of 
concurrency  between  initial  production  and  developmental  testing, 
beginning  production  before  demonstrating  that  a  system  will  work  as 
intended  increases  the  risk  of  deficiencies  that  require  substantial 
design  changes  and  costly  modifications  to  already-constructed 
systems. 

Eleven  of  these  programs  plan  to  complete  a  significant  portion  of 
their  testing — 30  percent  or  more — after  they  start  production.  When 
we  examined  the  relationship  between  concurrency  and  total  cost 
growth  for  programs  in  production,  we  found  that  as  concurrency 
increases,  so  does  the  total  acquisition  cost  growth.  We  also  found 
that  programs  with  higher  total  costs  tend  to  be  more  concurrent  than 
programs  with  lower  total  costs.  A  number  of  relatively  expensive  and 
concurrent  programs  have  yet  to  enter  production,  exposing  them  to 
increased  risk  of  design  changes  and  costly  retrofits.  Our  previous 
work  has  found  that  highly  concurrent  acquisition  strategies  can  result 
in  poor  cost,  schedule,  and  performance  outcomes.34 

Three  of  these  1 1  programs  plan  to  place  more  than  20  percent  of 
their  procurement  quantities  under  contract  before  testing  is  complete. 
For  example,  the  AIM-9X  Block  II  program  reported  that  it  had  about 
33  percent  of  its  total  procurement  quantity  under  contract,  at  a  cost  of 
approximately  $961  million,  before  completing  development  testing. 


oo 

GAO,  F-35  Joint  Strike  Fighter:  Problems  Completing  Software  Testing  May  Hinder 
Delivery  of  Expected  Warfighting  Capabilities,  GAO-14-322  (Washington,  D.C.:  Mar.  24, 
2014). 

34GAO,  Ford  Class  Aircraft  Carrier:  Poor  Outcomes  Are  the  Predictable  Consequences  of 
the  Prevalent  Acquisition  Culture,  GAO-16-84T  (Washington,  D.C.:  October  1, 2015). 
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Assessments  of 
Individual  Programs 


This  section  contains  assessments  of  individual  weapon  programs  which 
are  grouped  by  lead  service — Army,  Navy  and  Marine  Corps,  Air  Force, 
and  DOD-led — and  include  a  lead  service  separator  page  at  the  start  of 
each  grouping.  Each  assessment  presents  data  on  the  extent  to  which 
programs  are  following  a  knowledge-based  acquisition  approach  to 
product  development  and  other  program  information.  Each  lead  service 
separator  page  summarizes  information  about  the  acquisition  phase, 
current  estimated  funding  needs,  cost  and  schedule  growth,  and  product 
knowledge  attained  provided  in  the  assessments  that  follow.  In  total,  we 
present  information  on  55  programs.  For  42  programs,  we  produced  two- 
page  assessments  discussing  the  technology,  design,  and  manufacturing 
knowledge  obtained,  as  well  as  other  program  issues.  Each  two-page 
assessment  also  contains  a  comparison  of  total  acquisition  cost  from  the 
first  full  estimate  for  the  program  to  the  current  estimate.  The  first  full 
estimate  is  generally  the  cost  estimate  established  at  development  start; 
however,  for  a  few  programs  that  did  not  have  such  an  estimate,  we  used 
the  estimate  at  production  start  instead.  For  shipbuilding  programs,  we 
used  their  planning  estimates  if  those  estimates  were  available.  For 
programs  that  began  as  non-major  defense  acquisition  programs,  we 
used  the  first  full  estimate  available.  Thirty-nine  of  these  42  two-page 
assessments  are  of  major  defense  acquisition  programs,  most  of  which 
are  in  development  or  early  production,  and  three  assessments  are  of 
programs  that  were  projected  to  become  major  defense  acquisition 
programs  during  or  soon  after  our  review.  See  figure  14  for  an  illustration 
of  the  layout  of  each  two-page  assessment.  In  addition,  we  produced 
one-page  assessments  on  the  current  status  of  13  programs,  which 
include  12  future  major  defense  acquisition  programs  and  one  major 
defense  acquisition  program  that  is  well  into  production. 
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Figure  14:  Illustration  of  Program  Two-Page  Assessment 
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Leap  Component:  Army _ Common  Name:  AMF 

Airborne  and  Maritime/Fixed  Station  (AMF) 


The  Army's  AMF  program  plans  to  acquire  non- 
developmental  software-defined  radios--the  Small 
Airborne  Networking  Radio  (SANR)-and 
associated  equipment  for  integration  Into  Army 
rotary  wing  and  unmanned  aerial  systems.  The 
program  previously  planned  to  also  acquire  the 
Small  Airborne  Link  16  Terminal  (SALT)  radio,  but, 
In  August  2015,  the  Army  directed  program  officials 
to  dose-out  the  SALT  sub-program,  less  than  2 
years  after  it  was  established  We  assessed  SANR 
only. 


In  July  2012.  as  part  of  an  overall  Joint  Tactical 
Radio  System  (JTRS)  reorganization.  DOD 
directed  AMF  to  pursue  a  restructured  acquisition 
approach  and  acquire  the  desired  radios  as  a 
modified  non-developmental  item,  leveraging  to 
the  maximum  extent  practical  prior  investments 
made  since  2008  on  the  original  program  As  a 
result,  the  program  will  procure  existing  radios, 
which  will  be  tested  for  technology  maturity  as 
part  of  the  formal  testing  process.  Oue  to  the 
program's  non-developmental  strategy,  we  do  not 
have  Insight  Into  the  percentage  of  design 
drawings  released  and  officials  report  the  testing 
of  a  system-level  integrated  prototype  is  not 
applicable.  The  first  purchase  of  radios  is 
currently  scheduled  for  2018  with  full-rate 
production  scheduled  to  begin  in  fiscal  year  2020. 


0 


Common  Name:  AMF 


AMF  Program  ^  f 

Technology.  Design,  and  Production  Maturity 
In  July  2012-as  part  of  an  overall  JTRS 
reorganization  of  several  related  programs-the 
Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  directed  the  AMF 
program  to  pursue  a  restructured  acquisition 
approach  and  acquire  the  desired  radios  as  a 
modified  non-developmental  item,  leveraging  to  the 
maximum  extent  practical  prior  investments  made 
on  the  original  program  since  development  start  in 
2008.  This  lestructunng  shifted  the  program  from  a 
development  effort  supporting  Army,  Air  Force,  and 
Navy  platforms  to  one  that  supports  only  Army 
aviation  efforts.  While  program  officials  have 
identified  cnhcal  technologies,  the  program  intends 
to  procure  existing  radios,  which  will  be  tested  to 
demonstrate  technology  maturity  as  part  of  the 
overall  test  and  demonstration  process.  The 
program  does  not  intend  to  develop  any  new 
technologies  or  software  for  the  radio.  In  2014,  the 
Army  split  AMF  into  two  separate  sub-programs- 
SALT  and  SANR  In  August  2015,  the  Army  directed 
the  close-out  of  the  SALT  sub-program  and  no 
longer  intends  to  procure  these  radios,  as  they 
would  not  meet  the  Army's  operational  requirements 
until  fiscal  year  2021  and  are  more  expensive  than 
other  options 

The  SANR  sub  program  is  currently  in  the  pre¬ 
solicitation  phase  and  no  production  contracts  have 
been  awarded.  Program  officials  stated  that  the 
non-developmental  item  strategy  will  ensure  that  a 
certain  level  of  production  readiness  is  achieved 
They  added  that  fewer  than  100  radios,  out  of  the 
7.111  expected,  will  be  purchased  for 
developmental  testing  and  initial  platform 
integration  Reliability  venfication  testing  is 
expected  to  begin  after  the  purchase  of  the  first 
radios  in  fiscal  year  2018  and  will  be  completed 
pnor  to  the  start  of  full  rate  production  m  fiscal  year 
2020. 


The  government  plans  to  acquire  engineenng 
drawings  for  SANR  to  conduct  depot  level 
maintenance.  Program  officials  told  us  they  will  also 
assess  the  value  of  procuring  data  rights  for  future 
competitive  procurements,  which  we  have 
previously  reported  can  enable  the  government  to 


complete  maintenance  work  and  competitively 
award  contracts  for  further  acquisition  and 
sustainment 


Other  Program  Issues 

The  Army  approved  a  revised  acquisition  program 
baseline  for  AMF  in  May  2014.  Since  that  time, 
program  officials  told  us  that  the  research  and 
development  costs  for  SANR  have  increased  by 
5137.9  million.  They  stated  that  this  increase  has 
accelerated  the  program's  schedule.  Program 
officials  also  told  us  that  the  program's  procurement 
costs  increased  by  $54.4  million  They  attributed 
this  growth  to  the  addition  of  3  years  to  the  planned 
procurement  of  the  program,  from  1 1  to  14  years,  as 
a  result  of  a  warfighter  requirement 


Program  officials  told  us  that  they  plan  to  develop  a 
revised  acquisition  strategy  and  test  and  evaluation 
master  plan  for  the  program  In  fiscal  year  2016  to 
reflect  the  close-out  of  the  SALT  sub-program  They 
added  that  they  may  also  develop  a  revised 
acquisition  program  baseline  to  reflect  this  change 
Program  officials  stated  that  they  expect  to 
complete  both  an  affordability  analysis  and  should- 
cost  analysis  for  SANR  by  the  end  of  fiscal  year 
2016  to  support  the  future  production  contract 
award. 


Program  Office  Comments  ^ 

In  commenting  on  a  draft  of  this  report,  the  program 
office  noted  that  in  August  2015,  the  Army  closed 
out  the  SALT  sub-program  since  the  radios  would 
not  meet  the  Army's  operational  requirements  until 
fiscal  year  2021  and  would  result  in  a  more 
expensive  acquisition  strategy  than  other  options 
The  planned  total  quantity  of  690  radios  will  now 
leverage  a  current  interim  procurement  to 
accelerate  delivery  of  Link  16  capability  to  the  AH- 
64  E  platform  Currently,  the  product  manager  for 
AMF  Is  preparing  to  procure  SANR  using  a  non- 
developmental  item  acquisition  strategy,  which 
leverages  the  government’s  investments  in  software 
defined  radios,  and  relies  on  industry  to  fill  the  radio 
hardware  requirements.  Industry  will  use  existing 
government-owned  networking  waveforms.  A 
SANR  request  for  proposal  is  planned  to  be 
released  In  the  third  quarter  of  fiscal  year  2016. 


© 


GAO-K-32SSP  AtMMirwnU  of  Ma|of  W>«on  Proyi jm» 
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©  Program  description 

©  Illustration  or  photo  of  system 

0  Schedule  timeline  identifying  key  dates  for  the  program  including  the 
start  of  development,  major  design  reviews,  production  decisions, 
and  planned  operational  capability 

©  Program  Essentials  Programmatic  information  including  the  prime 
contractor,  program  office  location,  and  funding  needed  to  complete 

©  Program  Performance  Cost  and  schedule  baseline  estimates  and 
the  latest  estimate  provided  as  of  January  2016 


©  Brief  summary  describing  the  program’s  implementation  of 
knowledge-based  acquisition  practices  and  its  current  status 

0  Attainment  of  Product  Knowledge  Depiction  of  selected 
knowledge-based  practices  and  the  program’s  progress  in 
attaining  that  knowledge 

0  Assessment  of  program's  technology,  design,  and  production 
maturity,  as  well  as  other  program  issues 

O  Program  Office  Comments  General  comments  provided  by  the 
cognizant  program  office 


Source:  GAO  analysis.  |  GAO-16-329SP 

For  our  two-page  assessments,  we  depict  the  extent  of  knowledge  gained 
in  a  program  at  the  time  of  our  review  with  a  scorecard  and  narrative 
summary  at  the  bottom  of  the  first  page  of  each  assessment.  As 
illustrated  in  figure  14  above,  the  scorecard  displays  eight  key  knowledge- 
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based  acquisition  practices  that  should  be  implemented  by  certain  points 
in  the  acquisition  process.  The  more  knowledge  the  program  has  attained 
by  each  of  these  key  points,  the  more  likely  the  weapon  system  will  be 
delivered  within  its  estimated  cost  and  schedule.  A  knowledge  deficit 
means  the  program  is  proceeding  without  sufficient  knowledge  about  its 
technologies,  design,  or  manufacturing  processes,  and  faces  unresolved 
risks  that  could  lead  to  cost  increases  and  schedule  delays. 

For  each  program,  we  identify  a  knowledge-based  practice  that  has  been 
implemented  with  a  closed  circle.  We  identify  a  knowledge-based  practice 
that  has  not  yet  been  implemented  with  an  open  circle.  If  the  program  did 
not  provide  us  with  enough  information  to  make  a  determination,  we  show 
this  with  a  dashed  line.  A  knowledge-based  practice  that  is  not  applicable 
to  the  program  is  grayed  out.  A  knowledge-based  practice  may  not  be 
applicable  to  a  particular  program  if  the  point  in  the  acquisition  cycle  when 
the  practice  should  be  implemented  has  not  yet  been  reached,  or  if  the 
particular  practice  is  not  relevant  to  the  program.  For  programs  that  have 
not  yet  entered  system  development,  we  show  a  projection  of  knowledge 
attained  for  the  first  three  practices.  For  programs  that  have  entered 
system  development  but  not  yet  held  a  critical  design  review,  we  assess 
actual  knowledge  attained  for  these  three  practices.  For  programs  that 
have  held  a  critical  design  review  but  not  yet  entered  production,  we 
assess  knowledge  attained  for  the  first  five  practices.  For  programs  that 
have  entered  production,  we  assess  knowledge  attained  for  all  eight 
practices. 

We  make  adjustments  to  both  the  key  points  in  the  acquisition  cycle  and 
the  applicable  knowledge-based  practices  for  shipbuilding  programs.  For 
shipbuilding  programs  that  have  not  yet  awarded  a  detailed  design 
contract,  we  show  a  projection  of  knowledge  attained  for  the  first  three 
practices.  For  shipbuilding  programs  that  have  awarded  this  contract  but 
not  yet  started  construction,  we  would  assess  actual  knowledge  attained 
for  these  three  practices.  For  shipbuilding  programs  that  have  started 
construction,  we  assess  the  knowledge  attained  for  the  first  four 
practices.  We  do  not  assess  the  remaining  four  practices  for  shipbuilding 
programs,  as  they  are  not  applicable  for  these  programs.  See  figure  15 
for  examples  of  the  knowledge  scorecards  we  use  to  assess  these 
different  types  of  programs. 
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Figure  15:  Examples  of  Knowledge  Scorecards 

Program  in  production  Shipbuilding  program 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 

•  Demonstrate  all  critical  technologies  in  an  operational 
environment 

•  Complete  preliminary  design  review 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 


•  Test  a  system-level  integrated  prototype 


[  Manufacturing  processes  are  mature 

•  Demonstrate  critical  processes  are  in  control 

o 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

o 

•  Test  a  production-representative  prototype 

• 

^  Knowledge  attained  ■  ■  ■  ■ 

O  Knowledge  not  attained 

Information  not  available 

Not  applicable 

Attainment  of  Product  Knowledge 


As  of  January  2016 


Source:  GAO.  |  GAO-1 6-329SP 
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Statement  on 

Business 

Participation 


Small 


Pursuant  to  a  mandate  in  a  report  for  the  National  Defense  Authorization 
Act  for  Fiscal  Year  2013,  we  reviewed  whether  individual  subcontracting 
reports  from  a  program’s  prime  contractor  or  contractors  were  accepted 
on  the  Electronic  Subcontracting  Reporting  System  (eSRS).35  We 
reviewed  this  information  for  79  of  the  major  defense  acquisition 
programs  in  our  assessment  that  reported  contract  information  in  their 
December  2014  Selected  Acquisition  Report  (SAR)  submissions.  The 
contract  numbers  for  each  program’s  prime  contracts  were  entered  into 
the  eSRS  database  to  determine  whether  the  individual  subcontracting 
reports  from  the  prime  contractors  had  been  accepted  by  the  government. 
The  government  uses  individual  subcontracting  reports  on  eSRS  as  one 
method  of  monitoring  small  business  participation,  as  the  report  includes 
goals  for  small  business  subcontracting.  Not  all  prime  contracts  for  major 
defense  acquisition  programs  are  required  to  submit  individual 
subcontracting  reports.  For  example,  some  contractors  report  small 
business  participation  at  a  corporate  level  as  opposed  to  a  program  level, 
and  this  data  is  not  captured  in  the  individual  subcontracting  reports. 
Information  gathered  for  this  analysis  is  presented  in  appendix  VI. 


35H.R.  Rep.  No.  112-479,  at  284  (2012). 
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Army  Service  Summary 


We  performed  in-depth  assessments  on  10  of  the  15  Army  major  defense  acquisition  programs  in  the  current 
portfolio  that  for  the  most  part  are  in  system  development  or  the  early  stages  of  production.  We  also  assessed 
two  Army  programs  identified  as  future  major  defense  acquisition  progbrams,  which  are  expected  to  enter 
system  development  in  the  next  few  years.  The  Army  currently  estimates  a  need  of  more  than  $56  billion  in 
funding  to  complete  the  acquisition  of  these  programs.  The  programs  in  system  development  or  production, 
where  we  determined  cost  and  schedule  change  from  first  full  estimates,  have  experienced  a  net  cost 
decrease  of  almost  $17  billion  due  primarily  to  quantity  decreases  on  four  programs,  while  average  schedule 
delays  of  approximately  30  months  were  also  reported.  Of  these  same  programs,  one,  the  Armored  Multi- 
Purpose  Vehicle,  has  completed  all  the  activities  associated  with  the  applicable  knowledge-based  best 
practices  we  assess. 


Acquisition  Phase  and  Size  of  the 
12  Programs  Assessed 

Production 


System  development 

WIN-T 

Inc  ilBlt 


Technology  development 


IFPC  Inc  2-1 
Blk  1 


Currently  Estimated  Acquisition  Cost 
for  the  12  Programs  Assessed 


Fiscal  year  2016  dollars  in  billions 


Programs  in 

technology 

development 


Programs  in 

system 

development 


Cost  and  Schedule  Growth  on  8  Programs 
in  the  Current  Portfolio 

Fiscal  year  2016  dollars  in  billions 


Research  and  Procurement  Total 

development  costs  acquisition 

costs  costs 


Summary  of  Knowledge  Attained  to  Date  for 
Programs  Beyond  System  Development  Start 

Program 

common 

name 

Knowledge 
Point  (KP)  1 

Resources 

and 

requirements 

match 

Knowledge 
Point  2 

Product 
design 
is  stable 

Knowledge 
Point  3 

Manufacturing 
processes  are 
mature 

AMF 

• 

— 

KP  3 
in  future 

AMPV 

• 

KP  2 
in  future 

KP  3 
in  future 

CIRCM 

O 

KP  2 
in  future 

KP  3 
in  future 

HMS 

• 

• 

O 

IAMD 

o 

• 

KP  3 
in  future 

JAGM 

o 

• 

KP  3 
in  future 

M109A7  FOV 

• 

• 

O 

PAC-3  MSE 

• 

• 

O 

WIN-T  Inc  2 

• 

— 

• 

WIN-T  Inc  3 

• 

O 

•  All  applicable  knowledge  practices  were 
completed 

OOne  or  more  applicable  knowledge  practices 
were  not  completed 

All  knowledge  practices  were  not  applicable 
Information  not  available  for  one  or  more 
knowledge  practice 


□  Growth/reduction 
from  first  full  estimate 


□ 


First  full  estimate 


•  Cost  growth  of  more  than  15  percent  and/or 
schedule  delays  of  more  than  6  months 

•  Cost  growth  of  15  percent  or  less  and  schedule 
delays  of  6  months  or  less 

No  first  full  estimate  available 


Average 
acquisition 
cycle  time 

(in  months) 


Note:  Bubble  size  is  based  on  each  program’s  currently  estimated 
future  funding  needed. 


Note:  In  addition  to  research  and  development  and  procurement 
costs,  total  acquisition  cost  includes  acquisition  related  operations 
and  maintenance  and  system-specific  military  construction  costs. 


Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 
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Army  Program  Assessments 

Page 

2-page  assessments  Number 

Airborne  and  Maritime/Fixed  Station  (AMF)  63 

Armored  Multi-Purpose  Vehicle  (AMPV)  65 

Common  Infrared  Countermeasure  (CIRCM)  67 

Handheld,  Manpack,  and  Small  Form  Fit  Radios  (HMS)  69 

Integrated  Air  and  Missile  Defense  (IAMD)  71 

Joint  Air-to-Ground  Missile  (JAGM)  73 

M109A7  Family  of  Vehicles  (M109A7  FOV)  75 

Patriot  Advanced  Capability-3  Missile  Segment  Enhancement  (PAC-3  MSE)  77 

Warfighter  Information  Network-Tactical  (WIN-T)  Increment  2  79 

Warfighter  Information  Network-Tactical  (WIN-T)  Increment  3  81 

1-page  assessments 

Indirect  Fire  Protection  Capability  Increment  2-Intercept  Block  1  (IFPC  Inc  2-1  Block  1)  83 

Improved  Turbine  Engine  Program  (ITEP)  84 


Page  62 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


Lead  Component:  Army 


Common  Name:  AMF 


Airborne  and  Maritime/Fixed  Station  (AMF) 


The  Army's  AMF  program  plans  to  acquire  non- 
developmental  software-defined  radios-the  Small 
Airborne  Networking  Radio  (SANR)-and  associated 
equipment  for  integration  into  Army  rotary  wing  and 
unmanned  aerial  systems.  The  program  previously 
planned  to  also  acquire  the  Small  Airborne  Link  16 
Terminal  (SALT)  radio,  but,  in  August  2015,  the 
Army  directed  program  officials  to  close-out  the 
SALT  sub-program,  less  than  2  years  after  it  was 
established.  We  assessed  SANR  only. 


Source:  U.S.  Army. 


System  development  }>  Production 


▲ 
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End  operational 
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(3/08) 

(11/09) 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  TBD 

As  of 

Latest 

Percent 

Program  office:  Aberdeen  Proving 

10/2008 

09/2015 

change 

Ground,  MD 

Research  and  development  cost 

$2,086.7 

$1,510.5 

-27.6% 

Funding  needed  to  complete: 

Procurement  cost 

$6,661.3 

$1,625.3 

-75.6% 

R&D:  $77.6  million 

Total  program  cost 

$8,748.0 

$3,135.9 

-64.2% 

Procurement:  $1,625.3  million 

Program  unit  cost 

$0,323 

$0,220 

-31.7% 

Total  funding:  $1,702.9  million 

Total  quantities 

27,102 

14,222 

-47.5% 

Procurement  quantity:  14,060 

Acquisition  cycle  time  (months) 

80 

175 

118.8% 

The  quantities  identify  the  total  number  of  channels  required;  currently  one  SANR  radio  is  capable  of 

providing  two  channels. 

In  July  2012,  as  part  of  an  overall  Joint  Tactical 
Radio  System  (JTRS)  reorganization,  DOD 
directed  AMF  to  pursue  a  restructured  acquisition 
approach  and  acquire  the  desired  radios  as  a 
modified  non-developmental  item,  leveraging  to 
the  maximum  extent  practical  prior  investments 
made  since  2008  on  the  original  program.  As  a 
result,  the  program  will  procure  existing  radios, 
which  will  be  tested  for  technology  maturity  as 
part  of  the  formal  testing  process.  Due  to  the 
program's  non-developmental  strategy,  we  do  not 
have  insight  into  the  percentage  of  design 
drawings  released  and  officials  report  the  testing 
of  a  system-level  integrated  prototype  is  not 
applicable.  The  first  purchase  of  radios  is 
currently  scheduled  for  2018  with  full-rate 
production  scheduled  to  begin  in  fiscal  year  2020. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  ^ 

environment  ^ 

•  Demonstrate  all  critical  technologies  in  an  operational  £ 

environment 

•  Complete  preliminary  design  review  0 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 


•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


0  Knowledge  attained  ■* *■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  AMF 


AMF  Program 

Technology,  Design,  and  Production  Maturity 

In  July  2012-as  part  of  an  overall  JTRS 
reorganization  of  several  related  programs-the 
Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  directed  the  AMF 
program  to  pursue  a  restructured  acquisition 
approach  and  acquire  the  desired  radios  as  a 
modified  non-developmental  item,  leveraging  to  the 
maximum  extent  practical  prior  investments  made 
on  the  original  program  since  development  start  in 
2008.  This  restructuring  shifted  the  program  from  a 
development  effort  supporting  Army,  Air  Force,  and 
Navy  platforms  to  one  that  supports  only  Army 
aviation  efforts.  While  program  officials  have 
identified  critical  technologies,  the  program  intends 
to  procure  existing  radios,  which  will  be  tested  to 
demonstrate  technology  maturity  as  part  of  the 
overall  test  and  demonstration  process.  The 
program  does  not  intend  to  develop  any  new 
technologies  or  software  for  the  radio.  In  2014,  the 
Army  split  AMF  into  two  separate  sub-programs- 
SALT  and  SANR.  In  August  201 5,  the  Army  directed 
the  close-out  of  the  SALT  sub-program  and  no 
longer  intends  to  procure  these  radios,  as  they 
would  not  meet  the  Army's  operational  requirements 
until  fiscal  year  2021  and  are  more  expensive  than 
other  options. 

The  SANR  sub-program  is  currently  in  the  pre¬ 
solicitation  phase  and  no  production  contracts  have 
been  awarded.  Program  officials  stated  that  the 
non-developmental  item  strategy  will  ensure  that  a 
certain  level  of  production  readiness  is  achieved. 
They  added  that  fewer  than  1 00  radios,  out  of  the 
7,111  expected,  will  be  purchased  for 
developmental  testing  and  initial  platform 
integration.  Reliability  verification  testing  is 
expected  to  begin  after  the  purchase  of  the  first 
radios  in  fiscal  year  2018  and  will  be  completed 
prior  to  the  start  of  full-rate  production  in  fiscal  year 
2020. 

The  government  plans  to  acquire  engineering 
drawings  for  SANR  to  conduct  depot  level 
maintenance.  Program  officials  told  us  they  will  also 
assess  the  value  of  procuring  data  rights  for  future 
competitive  procurements,  which  we  have 
previously  reported  can  enable  the  government  to 


complete  maintenance  work  and  competitively 
award  contracts  for  further  acquisition  and 
sustainment. 

Other  Program  Issues 

The  Army  approved  a  revised  acquisition  program 
baseline  for  AMF  in  May  2014.  Since  that  time, 
program  officials  told  us  that  the  research  and 
development  costs  for  SANR  have  increased  by 
$137.9  million.  They  stated  that  this  increase  has 
accelerated  the  program's  schedule.  Program 
officials  also  told  us  that  the  program's  procurement 
costs  increased  by  $54.4  million.  They  attributed 
this  growth  to  the  addition  of  3  years  to  the  planned 
procurement  of  the  program,  from  11  to  14  years,  as 
a  result  of  a  warfighter  requirement. 

Program  officials  told  us  that  they  plan  to  develop  a 
revised  acquisition  strategy  and  test  and  evaluation 
master  plan  for  the  program  in  fiscal  year  2016  to 
reflect  the  close-out  of  the  SALT  sub-program.  They 
added  that  they  may  also  develop  a  revised 
acquisition  program  baseline  to  reflect  this  change. 
Program  officials  stated  that  they  expect  to 
complete  both  an  affordability  analysis  and  should- 
cost  analysis  for  SANR  by  the  end  of  fiscal  year 
2016  to  support  the  future  production  contract 
award. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  report,  the  program 
office  noted  that  in  August  2015,  the  Army  closed 
out  the  SALT  sub-program  since  the  radios  would 
not  meet  the  Army's  operational  requirements  until 
fiscal  year  2021  and  would  result  in  a  more 
expensive  acquisition  strategy  than  other  options. 
The  planned  total  quantity  of  690  radios  will  now 
leverage  a  current  interim  procurement  to 
accelerate  delivery  of  Link  16  capability  to  the  AH- 
64E  platform.  Currently,  the  product  manager  for 
AMF  is  preparing  to  procure  SANR  using  a  non- 
developmental  item  acquisition  strategy,  which 
leverages  the  government's  investments  in  software 
defined  radios,  and  relies  on  industry  to  fill  the  radio 
hardware  requirements.  Industry  will  use  existing 
government-owned  networking  waveforms.  A 
SANR  request  for  proposal  is  planned  to  be 
released  in  the  third  quarter  of  fiscal  year  2016. 
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Lead  Component:  Army 


Common  Name:  AMPV 


Armored  Multi-Purpose  Vehicle  (AMPV) 

The  Army's  Armored  Multi-Purpose  Vehicle  (AMPV) 
is  the  replacement  to  the  Ml  13  family  of  vehicles  at 
the  Brigade  level  and  below.  The  AMPV  will  replace 
the  Ml  13  in  five  mission  roles:  general  purpose, 
medical  evacuation,  medical  treatment,  mortar 
carrier,  and  mission  command.  The  Army 
determined  that  development  of  the  AMPV  is 
necessary  due  to  mobility,  survivability,  and  force 
protection  deficiencies  identified  with  the  Ml  13,  as 
well  as  space,  weight,  power,  and  cooling  limitations 
that  prevent  the  incorporation  of  future 
technologies. 

Source:  BAE  Systems. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  BAE  Systems 

As  of 

Latest 

Percent 

Program  office:  Warren,  Ml 

05/2015 

07/2015 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$1,004.0 

$1,031.2 

2.7% 

R&D:  $872.3  million 

Procurement  cost 

$9,891.9 

$9,887.2 

0.0% 

Procurement:  $9,886.8  million 

Total  program  cost 

$10,895.9 

$10,918.4 

0.2% 

Total  funding:  $10,759.1  million 

Program  unit  cost 

$3,711 

$3,719 

0.2% 

Procurement  quantity:  3,077 

Total  quantities 

2,936 

2,936 

0.0% 

Acquisition  cycle  time  (months) 

87 

87 

0.0% 

The  AMPV  program  entered  system  development 
in  December  2014  with  its  critical  technologies 
fully  mature.  The  program  held  its  preliminary 
design  review  in  June  2015,  6  months  after 
development  start.  The  AMPV  program  does  not 
intend  to  develop  new  technologies;  instead,  it 
plans  to  use  readily  available  components  and 
existing  technologies  from  legacy  systems. 
Although  the  program  did  not  conduct  competitive 
prototyping  prior  to  system  development,  the 
program  plans  to  employ  a  number  of  practices 
prior  to  its  critical  design  review  to  increase 
confidence  in  the  AMPV  design  and  to  preserve 
the  possibility  of  competition  in  later  phases  of  the 
program. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■*■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  AMPV 


AMPV  Program 

Technology  Maturity 

The  AMPV  program  entered  system  development  in 
December  2014  with  its  critical  technologies  fully 
mature  as  determined  by  an  independent  review 
team.  According  to  program  officials,  integration 
risks  are  low  to  moderate.  According  to  the  results 
of  the  program’s  preliminary  design  review,  software 
continues  to  be  a  risk  for  the  program.  Software 
development  for  the  AMPV  program  is  expected  to 
consist  of  80  percent  reuse  of  existing  software  and 
20  percent  new  development.  The  AMPV  program 
plans  to  reuse  software  under  development  for  an 
upgrade  to  another  system  and  previously 
developed  software  from  a  canceled  program.  The 
AMPV  software  schedule  is  dependent  on  the 
software  upgrade  being  developed  for  this  other 
system.  Any  significant  delay  with  this  software 
could  affect  the  AMPV  software  delivery  schedule 
and  delivery  of  development  vehicles. 

Design  and  Production  Maturity 

According  to  program  officials,  a  majority  of  the 
AMPV  design  is  derived  from  legacy  systems.  The 
program  held  a  preliminary  design  review  in  June 
2015,  6  months  after  development  start,  which 
resulted  in  43  action  items.  As  of  October  2015,  one 
action  item  remains  open.  According  to  a  Defense 
Contract  Management  Agency  report,  the  program's 
prime  contractor,  BAE  Systems,  entered  into  the 
preliminary  design  review  while  sub-systems  were 
undergoing  redesign.  However,  program  officials 
note  that,  in  their  experience,  it  is  not  uncommon  to 
refine  sub-system  design  between  the  preliminary 
and  critical  design  reviews,  as  the  detailed  design 
effort  includes  sub-system  design  trades  to  ensure 
requirements  are  met.  The  major  sub-systems  BAE 
redesigned  include  the  hull,  radiator,  battery,  and 
components  of  the  vehicle  propulsion  system. 
According  to  officials,  these  modifications  will 
provide  increased  soldier  force  protection,  power, 
reliability,  and  commonality  and  will  have  minimal 
impact  to  prototype  delivery  schedule.  The  program 
plans  to  use  other  knowledge-based  practices,  such 
as  identifying  critical  manufacturing  processes,  prior 
to  critical  design  review  to  increase  confidence  in 
the  stability  of  the  AMPV  design. 

The  Deputy  Assistant  Secretary  of  Defense, 
Systems  Engineering  identified  the  availability  of 
production  tooling  as  a  high  risk  area.  It  determined 


that  BAE  will  require  additional  tooling  mechanisms 
before  it  can  begin  producing  engineering  and 
manufacturing  development  prototype  vehicles  in 
June  2016.  The  program  expects  to  demonstrate 
critical  manufacturing  processes  on  a  pilot 
production  line  before  the  start  of  production,  but 
the  production  tooling  risk  might  impact  the 
program’s  ability  to  satisfy  this  best  practice  if  the 
required  tooling  is  unavailable.  Production  schedule 
delays  and  cost  growth  could  occur  if  the  required 
tooling  is  unavailable  by  the  start  of  production. 

Other  Program  Issues 

While  the  program  did  not  conduct  competitive 
prototyping  prior  to  the  start  of  system  development, 
the  program  plans  to  pursue  actions  to  maintain  the 
possibility  of  competition  at  the  prime  contractor 
level  before  the  completion  of  production. 
Specifically,  the  program  plans  to  acquire  complete 
technical  data  packages  and  may  use  modular, 
open  architectures  to  enable  competition  for 
upgrades.  Open  systems  architecture  allows  the 
product  to  be  refreshed  with  new,  improved 
components  made  by  a  variety  of  suppliers. 

Program  Office  Comments 

In  addition  to  providing  technical  comments,  the 
program  office  noted  that  the  risk  associated  with 
the  software  has  been  downgraded  as  the  software 
was  delivered  on  schedule  with  the  required 
functionality.  The  program  is  still  reliant  on  limited 
functionality  from  the  software  scheduled  to  be 
released  late  in  2015.  This  release  is  currently  on 
schedule,  and  upon  receipt  AMPV  will  no  longer  be 
dependent  on  other  systems’  software 
development.  The  identified  production  tooling  risk 
has  been  downgraded  since  preliminary  design 
review.  The  remaining  concern  is  final  development 
of  assembly  fixtures  that  may  cause  a  maximum  of 
a  30-day  schedule  slip. 

Six  months  after  the  scheduled  critical  design 
review,  BAE  Systems  will  begin  the  delivery  of  29 
system-level  prototypes  for  testing.  Testing  will 
consist  of  contractor  developmental  testing  and 
logistics  activities  along  with  a  government 
production  prove-out  test,  live  fire  test  and 
evaluation,  and  a  limited  user  test. 
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Lead  Component:  Army 


Common  Name:  CIRCM 


Common  Infrared  Countermeasure  (CIRCM) 

The  Army's  CIRCM  is  the  next  generation  of 
Advanced  Threat  Infrared  Countermeasures 
(ATIRCM)  designed  to  defend  aircraft  from  infrared- 
guided  missiles.  The  program  is  developing  a  laser- 
based  system  for  use  with  a  missile  warning  system 
and  countermeasure  dispenser  that  deploys 
expendables,  such  as  flares  and  chaff.  CIRCM  will 
be  installed  on  rotary-wing,  tilt-rotor,  and  small  fixed- 
wing  aircraft  across  DOD.  CIRCM  was  originally 
started  as  a  subprogram  under  the 
ATIRCM/Common  Missile  Warning  System. 


Source:  Northrop  Grumman. 
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Program  Essentials 

Prime  contractor:  Northrop  Grumman 
Program  office:  Huntsville,  Alabama 
35806 

Funding  needed  to  complete: 

R&D:  $452.7  million 
Procurement:  $1,804.6  million 
Total  funding:  $2,700.7  million 
Procurement  quantity:  1,076 


Program  Performance  (fiscal  year  2016  dollars  in  millions) 


As  of 

Latest 

11/2015 

Percent 

change 

Research  and  development  cost 

N/A 

$764.4 

N/A 

Procurement  cost 

N/A 

$1,804.6 

N/A 

Total  program  cost 

N/A 

$3,012.3 

N/A 

Program  unit  cost 

N/A 

$2,680 

N/A 

Total  quantities 

N/A 

1,124 

N/A 

Acquisition  cycle  time  (months) 

N/A 

115 

N/A 

The  CIRCM  program  entered  system 
development  in  August  2015  with  its  critical 
technologies  approaching  maturity  based  on  the 
results  of  a  technology  readiness  assessment 
conducted  in  December  2014.  Independent 
government  testing  of  system  performance 
validated  the  previous  vendor  testing, 
methodologies,  and  results.  The  review  team  was 
not  able  to  determine  that  any  of  the  critical 
technologies  had  achieved  full  maturity  as  testing 
of  the  CIRCM  prototypes  in  an  operational 
environment  was  not  performed.  The  program 
limited  competition  to  the  two  existing  contractors 
from  the  technology  development  phase.  The 
Army  also  plans  to  purchase  1 ,076  CIRCM  kits  for 
installation  on  aircraft. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  ^ 

environment  ^ 

•  Demonstrate  all  critical  technologies  in  an  operational  r 

environment 

•  Complete  preliminary  design  review  £ 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■* *■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  CIRCM 


CIRCM  Program 

Technology  Maturity 

The  CIRCM  program  entered  system  development 
in  August  2015  with  its  critical  technologies 
approaching  full  maturity.  An  independent  review 
team  conducted  an  assessment  of  the  program's 
nine  critical  technologies-including  the  gimbal 
assembly,  camera  assembly,  the  quantum  cascade 
laser,  and  others-and  determined  that  all 
technologies  were  nearing  maturity.  The  key  risks 
going  into  the  technology  development  phase  were 
CIRCM  weight,  probability  of  countermeasure,  and 
reliability.  According  to  the  acquisition  strategy,  key 
risks  to  meet  CIRCM  requirements  have  been 
significantly  reduced  in  the  technology  development 
phase.  A  combination  of  contractor  and  government 
testing  under  realistic  flight  conditions  has  allowed 
the  vendors  to  identify  weaknesses  and  address 
them  with  design  updates.  Government  test  results 
indicate  that  weight  and  probability  of 
countermeasure  have  met  exit  criteria  and/or 
system  requirements,  and  reliability  is  on  a  path  to 
meet  or  exceed  exit  criteria  and/or  the  system 
requirements. 

Other  Program  Issues 

Pursuant  to  a  recommendation  by  the  Office  of  the 
Secretary  of  Defense,  the  program  limited 
competition  to  the  two  existing  contractors  from  the 
technology  development  phase  to  avoid  duplication 
of  cost  and  development  efforts,  and  because 
market  research  indicated  that  only  the  current 
vendors  were  interested  in  the  development  effort. 
The  contract  type  for  system  development  was 
changed  to  also  include  fixed-price  incentive,  cost- 
plus-fixed-fee,  and  firm-fixed-price  terms. 

The  Undersecretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  approved  CIRCM's  entry 
into  the  engineering  and  manufacturing 
development  phase  on  August  25,  201 5.  The  Army 
was  authorized  to  proceed  with  the  award  of  the 
development  contract,  with  two  low-rate  initial 
production  options  for  up  to  30  modification  kits— 
which  include  the  hardware,  wiring  harness,  cables 
necessary  to  install  CIRCM  on  each  aircraft  but  not 
the  system  itself-and  45  hardware  kits  that  include 
the  CIRCM  systems,  37  for  the  Army  and  up  to  8  for 
the  Navy.  The  Army  plans  to  purchase  1,076 
CIRCM  systems  for  installation  on  aircraft.  It  will 
also  procure  3,373  modification  kits.  The  installation 


configuration  of  CIRCM  differs  from  platform  to 
platform.  CIRCM  is  not  intended  to  be  installed  on 
every  aircraft  but  instead  will  be  installed  on  select 
aircraft  based  on  the  mission  and  environment.  The 
Undersecretary  also  directed  the  Army  to  request 
funding  consistent  with  what  is  required  by  the 
independent  cost  estimate. 

The  Army  awarded  its  engineering  and 
manufacturing  development  contract  to  Northrop 
Grumman  on  August  28,  2015.  However,  the  British 
Aerospace  Electronics  Systems  filed  a  bid  protest 
on  September  8,  which  resulted  in  a  stop  work  order 
on  September  9.  The  bid  protest  was  resolved  on 
November  25  when  the  protester  withdrew  its 
protest  and  work  was  resumed  on  November  30, 
2015. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  stated  that  the  bid  protest  had  no 
major  adverse  effects  to  the  CIRCM  program's  cost 
or  performance.  However,  the  protest  impacted 
program  schedule,  resulting  in  delayed  fielding  to 
the  combat  aviation  brigades.  According  to  program 
officials,  until  CIRCM  is  fully  fielded,  upgrades  to  the 
ATIRCM  and  current  Army  Missile  Warning  System 
will  continue  to  provide  the  appropriate  coverage  to 
Army  aircrews.  The  program  office  also  provided 
technical  comments,  which  were  incorporated 
where  appropriate. 
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Lead  Component:  Army 


Common  Name:  HMS 


Handheld,  Manpack,  and  Small  Form  Fit  Radios  (HMS) 


DOD's  Joint  Tactical  Radio  System  (JTRS) 
program,  now  restructured,  was  developing 
software-defined  radios  to  interoperate  with  existing 
radios  and  increase  communications  and 
networking  capabilities.  The  Army’s  HMS  program 
carries  on  efforts  to  develop  two  radios  the  Rifleman 
radio  and  the  Manpack  radio.  A  subset  of  the 
Manpack  radios  are  designed  to  operate  with  the 
Mobile  User  Objective  System  (MUOS),  a  Navy 
satellite  communication  system  expected  to  serve  a 
worldwide,  multiservice  population  of  mobile  and 
fixed-site  terminal  users. 


Source:  ©  2012  General  Dynamics  C4S. 


System  development  Production 


▲ 

Program/ 
development  start 
(4/04) 


▲  ▲  ▲  ▲  ▲  ▲  ▲ 

Low-rate  Second  low-  Second  low-  Third  low-  GAO  Full-rate  Full-rate 

decision  rate  decision —  rate  decision —  rate  decision —  review  decision —  decision — 

(6/11)  Rifleman  radio  Manpack  radio  Manpack  radio  (1/16)  Rifleman  radio  Manpack  radio 


Program  Essentials 

Prime  contractor:  General  Dynamics  C4 
Systems,  Inc. 

Program  office:  Aberdeen  Proving 
Ground,  MD 

Funding  needed  to  complete: 

R&D:  $136.0  million 
Procurement:  $6,726.4  million 
Total  funding:  $6,862.5  million 
Procurement  quantity:  243,660 


(7/12)  (10/12)  (12/13)  (3/17)  (5/18) 


Program  Performance  (fiscal  year  2016  dollars  in  millions) 


As  of 

Latest 

Percent 

05/2004 

12/2014 

change 

Research  and  development  cost 

$5,84.4 

$1,395.2 

138.8% 

Procurement  cost 

$10,184.7 

$7,734.5 

-24.1% 

Total  program  cost 

$10,769.1 

$9,129.8 

-15.2% 

Program  unit  cost 

$0,033 

$0,034 

2.7% 

Total  quantities 

328,674 

271,202 

-17.5% 

Acquisition  cycle  time  (months) 

85 

124 

45.9% 

Since  our  last  assessment,  the  HMS  program- 
formerly  JTRS  HMS-has  demonstrated  all  its 
critical  technologies  as  mature  through  testing. 
However,  testing  to  verify  the  resolution  of  issues 
with  the  reliability  of  Manpack  and  suitability  of 
both  radios  remains  to  be  completed.  According 
to  officials,  the  HMS  program  will  conduct 
manufacturing  readiness  assessments  for  both 
the  Rifleman  and  Manpack  radios  as  part  of  the 
system’s  separate  full-rate  production  decisions 
scheduled  in  2017  and  2018,  respectively.  Testing 
associated  with  the  use  of  the  Navy's  MUOS 
waveform  in  some  of  the  radios  is  significantly 
delayed.  Additionally,  HMS  program  officials 
stated  that  changes  to  these  radios  necessary  for 
implementation  of  MUOS  required  the  program  to 
extend  the  software  development  effort. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


•  Release  at  least  90  percent  of  design  drawings  £ 


•  Test  a  system-level  integrated  prototype  £ 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control  9 

•  Demonstrate  critical  processes  on  a  pilot  production  line  o 

•  Test  a  production-representative  prototype  £ 

^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  HMS 


HMS  Program 

Technology  and  Design  Maturity 

As  of  July  201 5,  more  than  1 0  years  after  the  start 
of  development,  the  HMS  program  has 
demonstrated  all  of  its  critical  technologies  as 
mature  in  an  operational  environment.  Testing 
efforts  specific  to  the  critical  technologies  were 
completed  in  May  2015,  and  subsequent  testing 
efforts  are  devoted  to  systemic  testing  of  the  two 
radios.  The  program  office  also  reports  a  stable 
design,  with  some  additional  drawings  to  support 
redesign  work  in  response  to  past  testing  issues.  In 
fiscal  year  2014,  developmental  testing  identified 
concerns  with  the  Manpack  radios  reliability  and 
operational  testing  identified  suitability  concerns 
due  to  the  weight  of  the  unit.  Operational  testing 
also  identified  suitability  problems  with  the  Rifleman 
radio  due  to  overheating  and  rapid  battery 
depletion.  Program  officials  stated  that  redesign 
efforts  performed  by  the  contractor  have  resolved 
these  problems.  However,  additional  operational 
testing  may  be  necessary  to  ensure  that  all 
reliability  and  suitability  issues  with  the  two  radios 
have  been  addressed. 

Production  Maturity 

According  to  program  officials,  the  HMS  program 
conducted  manufacturing  readiness  assessments 
for  the  Rifleman  radios,  and  the  evaluation  of  these 
results  will  be  completed  in  advance  of  its  full-rate 
production  decision,  scheduled  for  March  2017. 
They  also  stated  that  the  HMS  program  will  conduct 
manufacturing  readiness  assessments  for  the 
Manpack  radios  vendors  as  part  of  its  separate  full- 
rate  production  decision,  scheduled  for  May  2018. 
However,  the  Manpack  assessments  have  not  yet 
been  conducted.  The  program  office  has  identified 
and  collected  specification  data  on  the  program’s 
critical  manufacturing  processes. 

Other  Program  Issues 

At  present,  use  of  the  MUOS  waveform  which 
some  Manpack  radios  will  rely  on  is  largely 
unavailable  to  the  warfighter  because  of  delays  with 
integrating  the  waveform,  radios,  and  ground 
system.  Although  MUOS  was  not  identified  as  a 
critical  technology,  without  this  waveform,  affected 
Manpack  radios  are  only  able  to  communicate 
through  legacy  communications  capabilities. 
According  to  program  officials,  the  development 
work  necessary  to  resolve  reliability  problems  with 


the  MUOS  waveform  has  been  completed,  and  the 
Navy  completed  the  first  phase  of  operational 
testing  of  the  MUOS  satellite  communication 
system  in  November  2015,  a  delay  of  17  months. 
Operational  testing  of  the  MUOS  waveform  is 
expected  to  be  fully  complete  by  June  2016.  The 
Army  plans  to  leverage  this  operational  testing  to 
inform  future  fielding  decisions.  According  to 
program  officials,  changes  to  the  Manpack  radios 
necessary  for  implementation  of  MUOS  required  the 
program  to  extend  software  development. 
Additional  software  development  to  add  new 
capabilities  associated  with  MUOS  is  now  expected 
to  continue  through  April  2016. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment, 
program  officials  stated  that  based  on  operator 
feedback  during  Manpack  radio  operational  testing, 
the  program  office  developed  an  improved 
backpack  which  provides  adequate  heat  dissipation 
and  weight  dispersion.  The  program  plans  to  deliver 
2,500  of  these  backpacks  in  2016.  The  program 
office  communicated  and  coordinated  with  industry 
to  synchronize  HMS  program  and  industry  partner 
capabilities  for  a  full  and  open  competition. 
Currently  in  source  selection,  contract  award  is 
expected  in  May  2016.  The  full-rate  production 
Manpack  radio  threshold  weight  is  3.5  pounds 
lighter  than  the  low-rate  production  radios,  and  will 
meet  the  8  hour  mission  requirement  using  a  single 
battery. 


Page  70 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


Lead  Component:  Army 


Common  Name:  IAMD 


Integrated  Air  and  Missile  Defense  (IAMD) 


The  Army's  Integrated  Air  and  Missile  Defense 
(IAMD)  program  is  being  developed  to  network 
sensors,  weapons,  and  a  common  battle  command 
system  across  an  integrated  fire  control  network  to 
support  the  engagement  of  air  and  missile  threats. 
The  IAMD  battle  command  system  will  provide  a 
capability  to  control  and  manage  IAMD  sensors  and 
weapons,  such  as  the  Sentinel  radar  and  Patriot 
launcher  and  radar,  through  an  interface  module 
that  supplies  battle  management  data  and  enables 
networked  operations. 


Source:  Northrop  Grumman. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

As  of 

Latest 

Percent 

Space  &  Mission  Systems  Corp, 

12/2009 

12/2014 

change 

Raytheon  Integrated  Defense  Systems 

Research  and  development  cost 

$1,711.4 

$2,600.9 

52.0% 

Program  office:  Redstone  Arsenal,  AL 

Procurement  cost 

$3,683.6 

$3,770.3 

2.4% 

Funding  needed  to  complete: 

Total  program  cost 

$5,395.0 

$6,371.1 

18.1% 

R&D:  $875.2  million 

Program  unit  cost 

$18,226 

$14,382 

-21.1% 

Procurement:  $3,770.3  million 

Total  quantities 

296 

443 

49.7% 

Total  funding:  $4,645.5  million 

Acquisition  cycle  time  (months) 

80 

102 

27.5% 

Procurement  quantity:  427 

IAMD  technologies  are  approaching  full  maturity 
and  at  least  90  percent  of  the  design  drawings 
have  been  released.  The  program  has 
encountered  software  integration  and 
synchronization  challenges,  but  according  to 
program  officials,  a  software  development  re-plan 
approved  in  early  2013  established  incremental 
deliveries  to  mitigate  these  challenges.  To  further 
reduce  risk,  the  program  rebaselined  its  schedule 
and  deferred  initial  operational  capability  by  2 
years.  IAMD  has  begun  developmental  flight  tests 
and  completed  the  first  two  test  flights.  Both  flight 
tests  were  assessed  as  having  met  all  test 
objectives,  including  communicating  within  the 
system,  tracking,  and  intercepting  the  target.  Two 
additional  flight  tests  are  planned  before  the  low- 
rate  initial  production  decision  scheduled  in 
August  2016. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  _ 

environment  w 

•  Demonstrate  all  critical  technologies  in  an  operational 

environment  *w 

•  Complete  preliminary  design  review  9 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings  £ 

•  Test  a  system-level  integrated  prototype  % 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  IAMD 


IAMD  Program 


Technology  Maturity 

lAMD's  four  critical  technologies-integrated  battle 
command,  integrated  defense  design,  integrated 
fire  control  network,  and  distributed  track 
management-are  not  expected  to  be  fully  mature 
until  the  program's  low-rate  initial  production 
decision  in  August  2016.  The  program  entered 
system  development  in  2009  with  these 
technologies  assessed  as  approaching  full  maturity 
based  on  a  notional  design. 


Program  Office  Comments 

The  program  office  was  provided  a  draft  of  this 
assessment  and  did  not  have  any  comments. 


Design  Maturity 

According  to  program  officials,  IAMD  is  dependent 
on  other  acquisition  programs  such  as  Patriot  and 
radar  programs  to  deliver  components  and  conduct 
testing.  As  system  integrator,  the  program  must 
coordinate  other  programs'  priorities  and  changes  to 
ensure  synchronization.  Although  the  program 
reports  that  all  of  the  expected  design  drawings 
have  been  released,  it  has  encountered  challenges 
integrating  and  synchronizing  new  software  from 
the  contractor  with  the  other  acquisition  programs 
IAMD  relies  on  for  its  functionality.  To  reduce 
integration  risks,  the  program  approved  a  software 
development  re-plan  in  April  2013,  which 
established  an  incremental  delivery  schedule  and 
updated  software  size  estimates.  According  to 
program  officials,  IAMD  is  receiving  software 
deliveries  to  support  the  test  program  and  will 
receive  updated  versions  of  the  software  throughout 
developmental  testing.  In  May  2015,  the  Army 
conducted  the  first  of  four  planned  flight  tests.  The 
overall  test  objective  was  to  demonstrate  the 
capability  of  the  system  to  receive  information  from 
sensors  and  communicate  information  to  the 
launcher  and  missile  to  attack  a  target  in  flight. 
According  to  officials,  the  first  flight  test  completely 
met  7  of  11  primary  objectives  and  substantially  met 
the  other  4.  The  test  also  included  two  secondary 
objectives  that  have  relevance  for  later  flights;  IAMD 
completely  met  one  and  partially  met  the  second.  A 
second  flight  test,  conducted  in  November,  also 
intercepted  the  target  and  demonstrated  the 
system’s  ability  to  identify,  track,  engage,  and  kill 
targets  using  an  interceptor  from  one  system  and 
remote  sensors  from  another  system  under  control 
of  the  IAMD  battle  command  system.  A  limited  user 
test  is  scheduled  for  completion  before  the  low-rate 
initial  production  decision  in  August  2016. 
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Lead  Component:  Army 


Common  Name:  JAGM 


Joint  Air-to-Ground  Missile  (JAGM) 

The  Joint  Air-to-Ground  Missile  is  an  Army-led 
program  with  joint  requirements  from  the  Navy  and 
Marine  Corps.  The  missile  is  designed  to  be  air- 
launched  from  helicopters  and  unmanned  aircraft 
systems  to  target  tanks;  light  armored  vehicles; 
missile  launchers;  command,  control  and 
communications  vehicles;  bunkers;  and  buildings.  It 
is  intended  to  provide  line-of-sight,  fire-and-forget, 
and  precision  attack  capabilities  in  day,  night,  and 
adverse  weather  conditions.  JAGM  will  replace  all 
HELLFIRE  missile  variants. 


Source:  U.S.  Army. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Lockheed  Martin 

As  of 

Latest 

Percent 

Program  office:  Redstone  Arsenal,  AL 

06/2015 

change 

Research  and  development  cost 

N/A 

$994.1 

N/A 

Funding  needed  FY  2016  to  FY  2020 

Procurement  cost 

N/A 

$562.4 

N/A 

R&D:  $200.9  million 

Total  program  cost 

N/A 

$1,556.6 

N/A 

Procurement:  $562.4  million 

Program  unit  cost 

N/A 

N/A 

N/A 

Total  funding:  $763  million 

Total  quantities 

N/A 

1,599 

N/A 

Procurement  quantity:  1,599 

Acquisition  cycle  time  (months) 

N/A 

38 

N/A 

The  latest  cost  and  quantity  data  reflect  only  the  funding  and  quantities  planned  for  fiscal  years  2008 

through  2020  and  not  the  cost  and  quantities  to  complete 

JAGM  entered  system  development  in  July  2015, 
after  extending  its  technology  development  by 
more  than  two  years,  and  had  not  yet 
demonstrated  its  critical  technologies  in  a  realistic 
environment.  A  preliminary  design  review  of  the 
fully  integrated  missile  was  not  conducted  prior  to 
development  start.  Instead,  the  program  relied  on 
component  level  reviews  and  reviews  conducted 
for  a  previous  design.  JAGM's  technologies  and 
design  are  expected  to  be  mature  by  its 
scheduled  critical  design  review  planned  in 
January  2016.  The  program  plans  to  manufacture 
JAGM  on  an  existing  production  line  used  for 
HELLFIRE  that  will  require  minor  modifications  to 
accommodate  the  longer  JAGM.  The  program 
has  not  yet  assessed  production  maturity,  but 
plans  to  demonstrate  the  new  processes  for 
JAGM  on  the  modified  production  line  and  assess 
manufacturing  processes  before  entering 
production. 


Attainment  of  Product  Knowledge 


Projected  as  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  _ 

environment  w 

•  Demonstrate  all  critical  technologies  in  an  operational  ^ 

environment  ^ 

•  Complete  preliminary  design  review  o 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings  £ 

•  Test  a  system-level  integrated  prototype  % 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 


Page  73 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


Common  Name:  JAGM 


JAGM  Program 

Technology  Maturity 

JAGM  has  three  critical  technologies-the  guidance 
seeker  assembly/sensor  platform,  sensor  software, 
and  mission  software.  The  sensor  software  is  fully 
mature  while  the  mission  software  and  guidance 
seeker  assembly/sensor  platform  are  projected  to 
reach  full  maturity  by  the  critical  design  review.  In 
2012,  JAGM  extended  its  technology  development 
period  by  more  than  two  years  and  determined  that 
requirements  could  be  met  with  existing 
technologies-a  HELLFIRE  motor,  warhead,  and 
electronics-coupled  with  a  newly  developed 
guidance  system.  In  July  2015,  the  Army  awarded  a 
system  development  contract  for  the  guidance 
system  to  Lockheed  Martin,  which  is  also  expected 
to  manufacture  the  fully  integrated  missile.  Program 
officials  identified  the  JAGM  software  development 
effort  as  a  risk  area.  The  program  uses  nine 
different  metrics  measuring  software  size, 
requirements,  testing,  and  defects  to  assess 
software  development  progress  and  maturity  with  all 
of  them  currently  meeting  expected  values. 

Design  Maturity 

A  system-level  preliminary  design  review  was 
completed  in  June  2010  on  a  JAGM  design 
developed  prior  to  the  program's  restructuring  in 
March  2012.  Design  changes  were  made  to  the 
missile  as  part  of  this  effort,  but  additional  reviews 
were  completed  only  for  the  guidance  subsystem 
and  not  the  re-designed,  fully  integrated  missile. 
DOD  waived  the  requirement  to  conduct  a  system- 
level  preliminary  review  of  the  new  design  before 
the  start  of  system  development  based  on  these 
prior  reviews  and  JAGM's  technology  maturity. 
Program  officials  report  that  the  current  design  effort 
consists  of  maturing  the  1 80  design  drawings  for  the 
guidance  subsystem,  177  of  which  are  complete. 
The  remaining  drawings  relate  to  the  subsystem's 
integration  with  the  missile.  According  to  program 
officials,  these  drawings  are  projected  to  be 
prepared  by  the  program's  critical  design  review, 
currently  scheduled  for  January  2016. 

Production  Maturity 

JAGM  will  be  manufactured  at  the  same  facility  as 
HELLFIRE-the  missile  JAGM  will  replace.  Only 
minimal  changes  are  expected  to  the  assembly  line 
to  accommodate  the  longer  JAGM.  For  example, 
new  holding  fixtures,  test  sets,  and  paint  equipment 


need  to  be  updated  to  accommodate  the  longer 
JAGM.  Production  maturity  has  not  yet  been 
assessed,  but  before  JAGM  enters  production  the 
program  office  plans  to  demonstrate  critical 
processes  on  the  modified  assembly  line  in  the  last 
quarter  of  fiscal  year  201 7.  In  addition  to 
demonstrating  and  qualifying  the  line  before 
production,  the  program  also  plans  to  assess  the 
extent  to  which  critical  manufacturing  processes  are 
under  statistical  control  and  reliably  yield  a  product 
that  meets  design  specifications. 

Other  Program  Issues 

The  Army  is  tracking  several  risks  that  could  affect 
cost  and  schedule.  One  risk  relates  to  the  in-flight 
reliability  requirement.  According  to  program 
officials,  there  must  be  no  more  than  two  failures  out 
of  the  48  JAGM  engineering  and  manufacturing  test 
flights.  If  more  than  two  failures  occur,  the  Army 
estimates  that  required  funding  could  increase  by  as 
much  as  10  percent,  and  the  schedule  would  be 
delayed  to  identify  the  root  cause  of  the  failures  and 
correct  them.  The  reliability  requirement,  however, 
is  being  reevaluated  and  may  be  descoped  pending 
Joint  Requirements  Oversight  Council  approval. 
The  program  is  also  tracking  risks  related  to 
schedule,  design  maturity,  and  production.  These 
risks,  according  to  the  program,  would  present 
minimal  or  no  impact  on  cost,  schedule,  or 
performance. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  indicated  that  the  JAGM  incremental 
strategy  leveraged  the  results  from  the  competitive 
prototyping  technology  development  phase  to 
develop  a  multi-mode  seeker  to  integrate  with  the 
HELLFIRE  backend.  The  approach,  according  to 
the  program,  was  an  effort  to  promote  fair 
competition.  Additionally,  not  testing  technologies  in 
a  realistic  environment  prior  to  development  start 
was  the  result  of  budget  constraints  on  the  program. 
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Common  Name:  M109A7  FOV 


M109A7  Family  of  Vehicles  (M109A7  FOV) 

The  Army's  M109A7  FOV  system  consists  of  two 
individual  platforms,  a  self-propelled  howitzer  (SPH) 
and  a  tracked  ammunition  carrier  that  provides 
operational  support.  The  SPH  is  a  tracked, 
aluminum  armored  vehicle  with  a  155  millimeter 
cannon.  The  M109A7  FOV  is  expected  to  provide 
improved  sustainability  over  the  current  howitzer 
fleet  through  the  incorporation  of  a  newly  designed 
hull;  modified  M2  Bradley  infantry  fighting  vehicle 
power  train,  suspension  system,  and  track;  and  a 
modernized  electrical  system. 


Source:  U.S.  Army. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  BAE  Systems  Land  & 

As  of 

Latest 

Percent 

Armament  L.P. 

02/2012 

06/2015 

change 

Program  office:  Warren,  Ml 

Research  and  development  cost 

$1,081.5 

$1,134.9 

4.9% 

Funding  needed  to  complete: 

Procurement  cost 

$6,094.1 

$6,066.0 

-0.5% 

R&D:  $194.4  million 

Total  program  cost 

$7,175.6 

$7,200.9 

0.4% 

Procurement:  $5,432.6  million 

Program  unit  cost 

$12,329 

$12,905 

4.7% 

Total  funding:  $5,627.0  million 

Total  quantities 

582 

558 

-4.1% 

Procurement  quantity:  521 

Acquisition  cycle  time  (months) 

118 

120 

1 .7% 

The  M109A7  FOV  program  entered  low-rate 
production  in  October  2013  with  its  two  critical 
technologies  mature  and  design  stable.  However, 
the  program  is  implementing  a  number  of  design 
changes  to  the  vehicle's  transmission,  fire  control 
system,  and  information  assurance  architecture 
that  will  result  in  retrofits  to  numerous  vehicles, 
cost  increases,  and  program  delays.  As  of  August 
2015,  the  program  office  indicated  that  a  number 
of  manufacturing  processes  and  systems  are  not 
fully  mature  and  the  capability  to  produce  the 
vehicles  has  not  yet  been  demonstrated.  The 
program  is  also  completing  developmental  testing 
concurrently  with  production.  This  concurrency 
between  testing  and  production  could  leave  it  at 
risk  for  additional  design  changes  and  costly 
retrofits  to  delivered  vehicles  if  further  deficiencies 
are  found  during  testing. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 

•  Demonstrate  all  critical  technologies  in  an  operational 
environment 

•  Complete  preliminary  design  review 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

• 

•  Test  a  system-level  integrated  prototype 

• 

1  Manufacturing  processes  are  mature 

•  Demonstrate  critical  processes  are  in  control 

o 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

o 

•  Test  a  production-representative  prototype 

• 

^  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 
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M109A7  FOV  Program 

Technology  and  Design  Maturity 

The  M109A7  FOV  program’s  two  critical 
technologies-power  pack  integration  and  the 
ceramic  bearing  of  the  generator  assembly-have 
been  assessed  as  fully  mature.  These  technologies 
are  identified  as  critical  based  on  concerns  about 
their  performance  at  high  temperatures.  While 
neither  technology  is  new,  failure  of  either  would 
represent  a  major  program  risk. 

According  to  program  officials,  the  design  is 
currently  stable  with  all  of  the  expected  drawings 
released  and  a  system-level  integrated  prototype 
demonstrated.  The  program  is,  however,  completing 
a  number  of  design  changes  that  will  result  in  at 
least  some  retrofits  to  all  67  low-rate  production 
vehicles.  Program  officials  indicated  that  the  current 
M109A7  transmission  will  be  replaced  at  full-rate 
production  by  a  more  efficient  transmission, 
resulting  in  retrofits  to  all  67  low-rate  production 
vehicles.  They  also  stated  that  the  M109A7’s 
current  steel  cannon  tube  will  also  be  replaced  with 
a  chrome  plated  design  to  meet  its  design  life,  which 
will  require  additional  retrofitting  for  those  vehicles 
produced  prior  to  July  2015.  Furthermore,  the 
current  M109A7  vehicle  architecture  for  information 
assurance  requires  soldiers  to  remove,  store,  and 
secure  modular  components,  thus  increasing 
operational  burden.  A  proposed  vehicle  architecture 
re-design  reduces  the  burden  to  soldiers  having  to 
complete  excessive  procedures  but  will  also  require 
retrofits.  Finally,  a  key  component  in  the  M109A7's 
fire  control  system  remains  a  risk  as  the  gun  can 
drift  off  target. 

Production  Maturity 

The  program  started  production  in  October  2013, 
and  the  first  vehicle  was  delivered  in  March  2015. 
To  assess  its  production  maturity,  the  program  uses 
metrics  related  to  labor  utilization,  quality  defects, 
and  manufacturing  readiness  levels.  Currently,  the 
program's  manufacturing  readiness  levels  for 
process  capability  and  control  indicate  that  its 
critical  production  processes  are  not  all  in  statistical 
control.  Our  best  practices  work  has  shown  that  if  a 
program's  critical  manufacturing  processes  are  not 
demonstrated  and  in  control  before  production 
begins,  it  is  at  risk  of  increased  cost  and  schedule. 
According  to  program  officials,  early  production 
results  showed  that  the  contractor  underestimated 


the  number  of  labor  hours  required  for 
manufacturing,  which  may  contribute  to  an 
estimated  $13  million  in  increased  costs  for  the  first 
production  lot. 

Other  Program  Issues 

The  program’s  current  schedule  has  a  limited 
amount  of  time  and  flexibility  to  correct  deficiencies 
identified  during  developmental  testing.  System- 
level  developmental  testing  is  expected  to  complete 
in  October  2016,  approximately  3  years  after 
production  start.  As  a  result,  the  Army  will  contract 
for  the  production  of  more  than  65  M109A7  FOV 
systems  before  completing  development  testing. 
The  program’s  estimated  cost  to  cover  the  retrofits 
related  to  problems  already  discovered  in  testing  is 
nearly  $20  million.  This  could  increase  if  additional 
deficiencies  are  found  during  the  testing  that  require 
design  changes  or  retrofits. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Patriot  Advanced  Capability-3  Missile  Segment  Enhancement  (PAC-3  MSE) 

The  Army's  PAC-3  MSE  is  a  surface-to-air  missile 
designed  to  defeat  tactical  ballistic  missiles  and 
other  aerial  threats.  Originally  a  subprogram  under 
the  PATRIOT/Medium  Extended  Air  Defense 
System  Combined  Aggregate  Program 
(PATRIOT/MEADS  CAP),  PAC-3  MSE  became  a 
separate  program  after  the  cancellation  of 
PATRIOT/MEADS  CAP  in  2013.  As  part  of  the 
PATRIOT  system,  which  includes  radars  and 
launchers,  PAC-3  MSE  improves  upon  earlier  PAC- 
3  variants  and  is  expected  to  provide  a  more  lethal 
interceptor  with  increased  battlespace  performance. 

Source:  U.S.  Army. 
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Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Lockheed  Martin 

As  of 

Latest 

Percent 

Program  office:  Redstone  Arsenal,  AL 

08/2004 

07/2015 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$579.2 

$961.7 

66.0% 

R&D:  $2.3  million 

Procurement  cost 

$7,238.9 

$5,383.3 

-25.6% 

Procurement:  $4,227.5  million 

Total  program  cost 

$7,818.1 

$6,354.4 

-18.7% 

Total  funding:  $4,239.1  million 

Program  unit  cost 

$5,117 

$5,814 

13.6% 

Procurement  quantity:  893 

Total  quantities 

1,528 

1,093 

-28.5% 

Acquisition  cycle  time  (months) 

N/A 

148 

N/A 

Change  in  acquisition  cycle  time  could  not  be  calculated  as  initial  estimates  did  not  include  an  initial 

operational  capability  date. 

The  program  entered  production  in  March  2014 
with  mature  technologies  and  a  stable  design,  but 
before  bringing  manufacturing  processes  under 
control.  Prior  to  its  production  decision,  the 
program  demonstrated  its  manufacturing 
processes  on  a  pilot  production  line  and  tested  a 
production-representative  prototype.  The 
PATRIOT  system  requires  additional  upgrades  to 
fully  test  and  utilize  all  the  PAC-3  MSE  capabilities 
prior  to  full-rate  production  of  the  missile.  This 
testing  was  delayed  over  a  year  due  to  limited 
access  to  PATRIOT  test  assets.  Further  schedule 
impacts  will  be  mitigated  by  a  dedicated  PATRIOT 
test  battalion  returning  from  deployment.  In  2015, 
the  program  definitized  its  production  contract 
after  several  delays,  accepted  delivery  of  the  first 
PAC-3  MSE  missiles,  received  approval  to 
conditionally  field  the  missiles,  and  equipped  its 
first  unit. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  _ 

environment  w 

•  Demonstrate  all  critical  technologies  in  an  operational  ^ 

environment  ^ 

•  Complete  preliminary  design  review  % 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

i 

• 

•  Test  a  system-level  integrated  prototype 

• 

1  Manufacturing  processes  are  mature 

•  Demonstrate  critical  processes  are  in  control 

o 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

•  Test  a  production-representative  prototype 

• 

9  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 
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PAC-3  MSE  Program 


Technology  and  Design  Maturity 

The  PAC-3  MSE  will  begin  operational  testing  in  the 
fourth  quarter  of  fiscal  year  201 6  with  mature 
technologies  and  a  stable  design.  While  the  PAC-3 
MSE  has  components  that  are  90  percent  common 
with  the  earlier  variant,  there  were  four  unique 
technologies  critical  to  PAC-3  MSE  that  required 
development:  a  dual  pulse  solid  rocket  motor, 
thermal  batteries,  an  ignition  safety  device,  and 
insensitive  munition  improvements  to  prevent 
inadvertent  launch  or  detonation. 

Production  Maturity 

PAC-3  MSE  began  production  in  March  2014  after 
testing  a  production-representative  prototype  and 
demonstrating  manufacturing  processes  on  a  pilot 
production  line,  but  before  demonstrating  critical 
processes  to  be  in  control.  Prior  to  its  production 
decision,  the  program's  manufacturing  readiness 
was  at  the  level  recommended  by  DOD  guidance, 
but  it  did  not  reach  the  level  that  indicated 
processes  were  in  control  as  recommended  by 
GAO  best  practices.  Program  officials  expect  to 
achieve  this  higher  level  prior  to  the  full-rate 
production  decision. 

Other  Program  Issues 

The  PAC-3  MSE  program  received  approval  to 
begin  producing  missiles  for  launch  from  the  current 
PATRIOT  system  software  and  hardware 
configuration.  However,  upgrades  to  this  software 
and  hardware  are  needed  to  fully  test  and  utilize  all 
the  PAC-3  MSE  capabilities  prior  to  full-rate  missile 
production.  Operational  testing  of  these  upgrades 
was  delayed  over  a  year  until  the  fourth  quarter  of 
fiscal  year  201 6  due  to  the  limited  availability  of  test 
assets  caused  by  the  operational  deployment  of  a 
PATRIOT  test  battalion  as  well  as  reallocation  of 
PATRIOT  test  assets  for  use  in  the  Army's 
Integrated  Air  and  Missile  Defense  program.  In 
August  2015,  the  program  office  was  able  to  fully 
address  the  equipment  and  manning  issues  when  it 
secured  a  dedicated  test  battalion,  returning  from 
deployment,  to  support  both  Integrated  Air  and 
Missile  Defense  and  PATRIOT  test  and  evaluation 
requirements.  The  program  plans  to  revise  its  test 
plan  by  the  second  quarter  of  fiscal  year  2016  to 
reflect  the  new  test  dates  leading  up  to  a  full-rate 
production  decision. 


In  October  2015,  the  PAC-3  MSE  program 
definitized  its  production  contract  after  several 
delays  and  then  completed  all  necessary  steps  to 
equip  its  first  PATRIOT  unit.  Production  of  the  PAC- 
3  MSE  began  under  an  undefinitized  contract  action 
issued  in  March  2014  as  a  short-term  vehicle  to 
allow  the  program  to  change  the  production  contract 
from  a  firm-fixed-price  to  a  fixed-price  incentive  (firm 
target)  contract.  This  change  was  also  in  alignment 
with  DOD's  Better  Buying  Power  initiatives. 
Agreement  on  final  contract  pricing  and  terms  was 
delayed  by  more  than  a  year  past  its  expected 
definitization  date  for  several  reasons,  including  the 
incorporation  of  changes  to  the  cost  analysis  and 
pricing  methodologies.  We  have  reported  in  the  past 
that  delays  in  finalizing  this  type  of  contract  places 
the  program  at  risk  of  cost  growth,  as  the 
government  normally  reimburses  contractors  for  all 
allowable  costs  incurred  during  the  undefinitized 
period,  giving  contractors  little  incentive  to  control 
costs.  In  October  2015,  the  program  definitized  the 
contract,  received  the  first  12  missiles  from  the 
contractor,  and  received  approval  for  conditional 
fielding  of  the  missiles.  By  the  end  of  the  month,  the 
program  had  successfully  equipped  the  first 
PATRIOT  unit  with  these  missiles. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
Army  provided  technical  comments,  which  were 
incorporated  as  appropriate. 
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Warfighter  Information  Network-Tactical  (WIN-T)  Increment  2 

WIN-T  is  the  Army's  high-speed  and  high-capacity 
tactical  communications  network.  It  connects  units 
with  higher  levels  of  command  and  is  being  fielded 
in  several  increments.  The  second  increment  is 
expected  to  provide  the  Army  with  required 
networking  on-the-move  capability.  The  third 
increment  is  now  a  software-only  development  effort 
that  will  provide  critical  software  upgrades  to 
support  improved  network  capabilities  for  the  first 
and  second  increments. 


Source:  U.S.  Army. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  General  Dynamics  C4 

As  of 

Latest 

Percent 

Systems,  Inc. 

10/2007 

09/2015 

change 

Program  office:  Aberdeen  Proving 

Research  and  development  cost 

$256.0 

$287.4 

12.3% 

Ground,  MD 

Procurement  cost 

$3,722.5 

$10,060.1 

170.2% 

Funding  needed  to  complete: 

Total  program  cost 

$3,978.5 

$10,347.5 

160.1% 

R&D:  $4.2  million 

Program  unit  cost 

$2,102 

$2,774 

32.0% 

Procurement:  $7,157.5  million 

Total  quantities 

1,893 

3,730 

97.0% 

Total  funding:  $7,161.7  million 

Acquisition  cycle  time  (months) 

50 

74 

48.0% 

Procurement  quantity:  2,372 

WIN-T  Increment  2  entered  production  in  March 
2010  with  its  critical  technologies  mature  and, 
according  to  program  officials,  a  stable  design. 
The  program  had  struggled  to  demonstrate  the 
required  performance  and  reliability  during  earlier 
rounds  of  operational  testing,  and  a  third 
operational  test  concluded  in  November  2014 
identified  that,  while  the  program  has 
demonstrated  continued  reliability  growth,  certain 
deficiencies  remained.  The  program  reported  a 
Nunn-McCurdy  unit  cost  breach  of  the  significant 
threshold  in  January  2015  due  to  changes  in  the 
relative  mix  of  items  procured  and  a  10  year 
schedule  extension.  In  June  2015,  DOD  approved 
the  program  to  start  full-rate  production,  assigned 
affordability  caps  on  unit  costs,  and  directed  the 
program  to  take  actions  to  address  performance 
issues. 


Attainment  of  Product  Knowledge 


As  of  January  2016 
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Common  Name:  WIN-T  Increment  2 


WIN-T  Increment  2  Program 

Technology,  Design,  and  Production  Maturity 

All  15  WIN-T  Increment  2  critical  technologies  were 
mature  by  its  March  2010  production  start.  The 
program  office  does  not  track  the  metric  we  use  to 
measure  design  maturity-the  number  of  releasable 
drawings-as  WIN-T  is  primarily  an  information 
technology  integration  effort.  Instead,  design 
performance  is  measured  through  a  series  of 
component,  subsystem,  configuration  item,  and 
network-level  tests  designed  to  demonstrate 
performance  at  increasing  levels  of  system 
integration.  According  to  the  WIN-T  program,  it  has 
integrated  and  tested  its  key  technologies  and 
subsystems,  demonstrating  that  the  system's 
design  is  capable  of  working  as  intended  and  is 
mature.  The  program  began  testing  a  production- 
representative  prototype  in  March  2011  and 
assessed  its  manufacturing  processes  as  in  control 
in  2012,  both  after  production  began. 

Other  Program  Issues 

The  program  struggled  to  demonstrate  the  required 
performance  and  reliability  during  earlier  rounds  of 
operational  testing.  A  third  operational  test,  which 
concluded  in  November  2014,  showed  improved 
reliability  and  performance  compared  to  prior  tests 
but  found  that  deficiencies  remain.  In  particular, 
while  many  of  the  program's  components  have  now 
been  assessed  as  operationally  effective  and 
suitable,  two  were  assessed  as  not  operationally 
effective  due  to  the  limits  of  one  waveform  used  for 
communications.  The  Stryker  variants  of  two  other 
components  were  assessed  as  not  operationally 
suitable  due  to  significant  integration  and  user 
issues  that  interfered  with  a  unit's  ability  to  operate. 
The  most  recent  operational  tests  also  found  that 
the  program  has  significant  cyber  security 
deficiencies.  As  a  result,  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics 
directed  the  Army  to  perform  an  independent  cyber 
design  and  implementation  assessment.  Cyber 
security  vulnerabilities  reduced  the  program’s 
operational  capacity  and  could  incur  cost  growth 
and  schedule  delays  if  significant  changes  are 
necessary.  According  to  program  officials,  the 
independent  assessment  is  complete  and  was 
provided  to  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  in  October 
2015. 


In  January  201 5,  the  Army  announced  that  the 
program  had  experienced  a  significant  Nunn- 
McCurdy  cost  breach  due  to  increased  unit  costs. 
The  Army  stated  that,  as  a  result  of  changes  in  the 
Army  network  modernization  strategy,  there  have 
been  changes  in  the  relative  mix  of  items  it  is 
procuring  to  include  a  higher  percentage  of  more 
expensive  items.  In  addition,  the  WIN-T  Increment  2 
procurement  schedule  was  extended  by  10  years  to 
support  fielding  requirements  as  part  of  the  WIN-T 
Increment  3  restructure.  Program  officials  told  us 
future  unit  cost  growth  remains  an  ongoing  concern. 
The  program  could  be  at  risk  of  an  additional  unit 
cost  breach  if  there  are  significant  quantity  changes 
in  the  future. 

The  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  approved  the  program  to 
enter  full-rate  production  in  June  2015.  He  also 
directed  the  program  to  take  corrective  actions  to 
address  the  performance  deficiencies  and  cyber 
security  vulnerabilities  discovered  in  testing  and 
established  affordability  caps  on  the  program's  unit 
costs.  According  to  program  officials,  each  of  these 
corrective  actions  has  either  been  completed  or  is 
under  way. 

Program  Office  Comments 

In  addition  to  providing  technical  comments,  the 
program  office  stated  it  has  developed  corrective 
action  plans  to  address  issues  identified  in  the  full- 
rate  acquisition  decision  memorandum-Stryker 
integration  and  limitations  of  one  waveform  used  for 
communications.  Testing  has  demonstrated 
improvements  in  cyber  security,  but  certain  issues 
remain.  The  program  is  addressing  and  executing 
cyber  security  issues  with  defined  short,  mid  and 
long-term  mitigation  steps.  The  Army  continues 
fielding  to  priority  units-a  total  of  5  divisions  and  14 
brigade  combat  teams  have  been  fielded  through 
early  fiscal  year  2016,  and  the  program  is  now 
making  improvements  to  network  initialization  and 
integration  of  WIN-T  systems  to  help  enable 
airborne  operations. 
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Lead  Component:  Army 


Common  Name:  WIN-T  Increment  3 


Warfighter  Information  Network-Tactical  (WIN-T)  Increment  3 

WIN-T  is  the  Army's  high-speed  and  high-capacity 
communications  network.  It  connects  units  with 
higher  levels  of  command  and  is  being  fielded  in 
several  increments.  Increment  3,  the  increment 
assessed  here,  provides  software  enhancements  to 
the  Army's  communication  for  improved  network 
capacity  and  robustness.  This  software  will  be  used 
to  update  and  enhance  hardware  procured  for  the 
first  and  second  increments. 


Source:  U.S.  Army. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  General  Dynamics  C4 

As  of 

Latest 

Percent 

Systems,  Inc. 

05/2009 

09/2015 

change 

Program  office:  Aberdeen  Proving 

Research  and  development  cost 

$2,883.2 

$2,024.6 

-29.8% 

Ground,  MD 

Procurement  cost 

$14,677.2 

$0.0 

-100.0% 

Funding  needed  to  complete: 

Total  program  cost 

$17,560.4 

$2,024.6 

-88.5% 

R&D:  $38.7  million 

Program  unit  cost 

$5,043 

N/A 

N/A 

Procurement:  $0.0  million 

Total  quantities 

3,482 

0 

-100.0% 

Total  funding:  $38.7  million 

Acquisition  cycle  time  (months) 

165 

N/A 

N/A 

Procurement  quantity:  0 

In  May  2014,  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  approved 
the  Army's  request  to  restructure  the  WIN-T 
Increment  3  program,  which  eliminated  the 
requirements  for  hardware  but  retained  the 
software  development  efforts  to  update  WIN-T 
Increments  1  and  2.  As  a  result,  the  program  is 
reporting  fewer  critical  technologies,  and  the 
remaining  technologies  are  intended  to  improve 
network  operations  and  increase  the  throughput 
available  for  satellite  communications.  The  Army 
plans  to  continue  software  development  and 
testing  through  fiscal  year  2016.  Upon 
completion,  WIN-T  Increment  3  will  cease  to  be 
an  independent  acquisition  program,  and  the 
WIN-T  Increment  1  and  Increment  2  programs  will 
complete  the  software  integration,  operational 
testing,  and  fielding. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  _ 

environment  w 

•  Demonstrate  all  critical  technologies  in  an  operational  ^ 

environment  ^ 

•  Complete  preliminary  design  review  % 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  WIN-T  Increment  3 


WIN-T  Increment  3  Program 

Technology  Maturity 

As  we  previously  reported,  the  Army  restructured 
the  program  in  2014  and  descoped  the  technologies 
associated  with  hardware  development,  including 
the  digital  transceiver  and  antennas.  As  a  result, 
WIN-T  Increment  3  is  now  a  software-only 
development  program,  with  development  and 
testing  continuing  through  fiscal  year  2016.  While 
the  remaining  technologies  improve  network 
operation  by  simplifying  network  management, 
increasing  automation,  and  boosting  throughput  and 
robustness,  the  current  program  development  path 
will  eliminate  the  expected  increase  in  network 
density  allowed  by  descoped  hardware.  According 
to  program  officials,  by  eliminating  aircraft-based 
antenna  hardware  more  data  will  have  to  be 
transmitted  via  satellite,  which  is  both  expensive 
and  can  be  interrupted  via  jamming.  Officials  also 
stated,  however,  that  the  requirement  for  this 
capability  has  been  deferred  due  to  the  cost,  given 
the  current  budget  environment,  rather  than 
eliminated  completely.  The  program  office  considers 
its  remaining  technologies  mature.  Some  of  the 
critical  technologies  may  be  replaced  by  backup 
technologies,  however,  the  decision  on  whether  or 
not  to  do  so  will  not  occur  until  after  a  Network 
Integration  Exercise  currently  scheduled  for  spring 
2016.  The  WIN-T  Increment  1  and  Increment  2 
programs  will  complete  the  integration,  operational 
testing,  and  fielding  of  the  WIN-T  Increment  3 
software. 

Design  and  Production  Maturity 

The  program  office  does  not  track  the  metric  we  use 
to  measure  design  maturity-the  number  of 
releasable  drawings  as  WIN-T  Increment  3  is  a 
software-only  development  program.  Instead, 
design  performance  is  measured  using  earned 
value  management  data  that  tracks  the  progress  of 
work  completed  against  projected  cost  and 
schedule  estimates.  According  to  the  program 
office,  it  is  not  meeting  the  projected  estimates. 
Officials  stated  that  they  also  use  incident  reports 
and  the  closure  rate  of  those  reports  to  track 
progress.  With  the  elimination  of  WIN-T  Increment 
3's  unique  hardware  development  no  production  is 
required. 


Other  Program  Issues 

According  to  program  documentation,  the 
restructuring  of  the  WIN-T  program  was  due  to 
fiscal  constraints  that  forced  the  Army  to  adjust  its 
funding  priorities  and  requirements.  The  program 
will  continue  submitting  defense  acquisition 
executive  summaries  and  selected  acquisition 
reports  until  its  capabilities  are  incorporated  into 
WIN-T  Increments  1  and  2.  After  the  program 
provides  the  planned  capabilities  in  fiscal  year  201 6, 
it  will  cease  to  be  an  independent  acquisition 
program. 

Program  Office  Comments 

The  program  office  concurred  with  this  assessment 
and  provided  technical  comments,  which  were 
incorporated  where  appropriate. 
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Lead  Component:  Army 


Common  Name:  IFPC  Inc.  2-1  Block  1 


Indirect  Fire  Protection  Capability  Increment  2-Intercept  Block  1 

(IFPC  Inc.  2-1  Block  1) 

The  Army’s  IFPC  Inc.  2-1  is  a  follow-on  effort  to 
enhance  and  extend  the  range  of  the  first  IFPC 
increment  fielded  in  2004,  which  provided  a  short- 
range  capability  to  counter  threats  from  rockets, 
artillery,  and  mortars.  IFPC  Inc.  2-1  is  being 
developed  in  three  blocks,  each  a  separate  major 
defense  acquisition  program.  Block  1  adds  the 
capability  to  counter  cruise  missiles  and  unmanned 
aircraft.  The  remaining  blocks  enhance  and  extend 
the  range  of  IFPC’s  capabilities  to  defend  against 
these  various  threats.  We  assessed  Inc.  2-1  Block  1. 


Source:  IFPC  Inc  2  Product  Office. 


Current  Status 

The  IFPC  Inc.  2-1  Block  1  program  is  developing  a  multi-mission  launcher  (MML)  capable  of  launching  at 
least  six  different  types  of  missiles.  According  to  program  officials,  the  Army  intends  to  leverage  existing 
sensors,  interceptors,  and  network  capabilities  and  integrate  them  with  the  launcher  to  deliver  a  system-of- 
systems  air  defense  capability.  Based  on  the  Army’s  determination  that  in-house  development  of  the  MML  is 
estimated  to  cost  less  than  developing  prototypes  through  one  or  more  contractors,  all  development  will  take 
place  at  two  Army  facilities,  the  Aviation  and  Missile  Research  Development  and  Engineering  Center 
(AMRDEC)  and  Letterkenny  Army  Depot  (LEAD).  The  Army  has  completed  the  assembly  of  two  MML 
prototypes  and  is  currently  integrating  them  with  the  other  existing  systems  necessary  for  deployment  of  an 
initial  operational  capability.  The  program  entered  technology  development  in  March  2014  and  a  year  later 
successfully  launched  three  different  types  of  missiles  from  its  prototype.  In  March  2016,  the  program  plans 
to  assess  the  system’s  maturity  by  launching  multiple  types  of  missiles  from  the  MML  during  an  engineering 
demonstration.  If  this  demonstration  is  successful,  according  to  program  officials,  all  technologies  will  be  at 
least  approaching  full  maturity  by  the  start  of  system  development. 

According  to  program  officials,  the  IFPC  Inc.  2-1  Block  1  program  held  a  successful  preliminary  design 
review  in  September  2015  and  was  granted  approval  to  proceed  to  a  system  development  start  in  June 
2016  and  critical  design  review  in  September  2016.  In  January  2015,  the  program  completed  a  cost-benefit 
analysis  to  determine  a  strategy  for  development.  Among  the  alternatives  reviewed  were  continuing  the 
exclusive  use  of  Army  facilities,  full  and  open  competition  among  contractors,  a  public-private  partnership 
between  the  Army  facilities  and  a  contractor,  or  some  combination.  The  analysis  concluded  that  continuing 
to  use  the  Army’s  facilities  at  AMRDEC  and  LEAD  exclusively  would  be  the  least  expensive  alternative.  The 
program  is  required  to  conduct  a  similar  analysis  to  evaluate  strategies  for  production. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2016  dollars): 

Total  program:  $2.6  billion 

Research  and  development:  $522  million 

Procurement:  $2.1  billion 

Quantity:  8  (development),  360  (procurement),  368  (total) 

Next  Major  Program  Event:  System  development  start,  June  2016 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 
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Lead  Component:  Army 


Common  Name:  ITEP 


Improved  Turbine  Engine  Program  (ITEP) 


The  Army’s  Improved  Turbine  Engine  Program 
(ITEP)  is  developing  a  replacement  engine  for  the 
Black  Hawk  and  Apache  helicopter  fleets.  The  new 
engine  is  designed  for  increased  power, 
performance,  and  fuel  efficiency;  enhanced 
reliability;  increased  service  life;  and  a  lower 
maintenance  burden  than  current  engines.  The 
increased  fuel  efficiency  is  expected  to  lessen  the 
need  for  refueling  by  providing  increased  operational 
range.  The  Army  plans  to  begin  fielding  these 
engines  in  fiscal  year  2027 


Source:  U.S.  Army. 


Current  Status 

In  November  2012,  ITEP  entered  the  material  solution  analysis  phase  and  conducted  an  analysis  of 
alternatives.  During  that  phase,  two  contractors  developed  component  prototypes  to  mature  technologies, 
design  and  manufacture  prototype  components,  perform  component  assembly,  and  conduct 
demonstrations.  Subsequently,  an  independent  review  team  determined  that  the  three  critical 
technologies — advanced  inlet  particle  separator,  compressor/advanced  aerodynamics,  and  hybrid 
bearings — were  approaching  full  maturity. 

In  June  2015,  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  approved  the 
program’s  entry  into  the  technology  maturation  and  risk  reduction  phase  as  a  major  defense  acquisition 
program.  ITEP  plans  to  award  two  fixed-price  incentive  (firm  target)  preliminary  design  review  contracts  to 
establish  the  baseline  of  the  engine  system  and  sub-elements,  obtain  information  regarding  cost,  and  further 
demonstrate  technologies.  According  to  officials,  the  expected  value  of  these  contracts  represents 
approximately  one-fifth  of  the  program’s  total  estimated  research  and  development  cost. 

The  Army  has  identified  moderate  risks  associated  with  increased  life  cycle  costs,  key  performance 
parameters,  and  platform  integration  requirements.  However,  mitigation  plans  are  in  place  and  the  Army 
does  not  expect  them  to  impact  system  development.  Following  completion  of  the  planned  preliminary 
design  review  in  the  first  quarter  of  fiscal  year  2018,  the  Army  intends  to  award  a  cost-plus-incentive-fee 
contract  for  system  development.  Competition  for  this  contract  will  be  limited  to  the  two  contractors  awarded 
preliminary  design  contracts. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2016  dollars): 

Total  program:  $10,011.0  million 

Research  and  development:  $1,852.9  million 

Procurement:  $8,158.1  million 

Quantity:  68  (development),  6,215  (procurement) 

Next  Major  Program  Event:  Technology  maturation  and  risk  reduction  start,  April  2016 

Program  Office  Comments:  According  to  the  Army,  preliminary  design  request  for  proposals  was  released 
in  September  2015  with  an  anticipated  award  in  fourth  quarter  fiscal  year  2016.  In  addition,  technical 
comments  were  incorporated  as  appropriate. 
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Navy  and  U.S.  Marine  Corps  Service  Summary 


We  performed  in-depth  assessments  on  15  of  the  35  Navy  and  Marine  Corps  programs  in  the  current  portfolio 
currently  between  the  start  of  development  and  the  early  stages  of  production.  In  addition,  we  assessed  the 
Amphibious  Combat  Vehicle  which  began  system  development  in  November  2015  and  another  seven 
programs  identified  as  future  major  defense  acquisition  programs  which  are  expected  to  enter  system 
development  in  the  next  few  years.  The  Navy  and  Marine  Corps  currently  estimate  a  need  of  more  than  $213 
billion  in  funding  to  complete  the  acquisition  of  these  23  programs.  The  programs  currently  in  the  portfolio  for 
which  we  determined  cost  and  schedule  change  from  first  full  estimates  have  experienced  approximately  $117 
billion  in  cost  growth,  the  majority  of  which  occurred  more  than  five  years  ago,  and  average  schedule  delays  of 
40  months.  Of  the  programs  in  the  current  portfolio  where  a  two-page  assessment  was  prepared,  two  have 
completed  all  the  activities  associated  with  the  applicable  knowledge  based  best  practices  we  assess. 


Acquisition  Phase  and  Size  of  the 
23  Programs  Assessed 


Currently  Estimated  Acquisition  Cost 
for  the  23  Programs  Assessed 
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No  first  full  estimate  available 


Note:  Bubble  size  is  based  on  each  program’s  currently  estimated 
future  funding  needed. 


Summary  of  Knowledge  Attained  to  Date  for 
Programs  Beyond  System  Development  Start 


Programs 
in  system 
development 


Programs  in 
production 


Program 

common 

name 

Knowledge 
Point  (KP)  1 

Resources 

and 

requirements 

match 

Knowledge  Knowledge 
Point  2  Point  3 

Product  Manufacturing 

design  processes  are 

is  stable  mature 

AIM-9X 
Block  II 

• 

• 

• 

AMDR 

O 

• 

KP  3 
in  future 

ACV 

O 

O 

KP  3 
in  future 

CH-53K 


VH-92A 


O 


KP  3 
in  future 


DDG  1000 

o 

• 

CVN  78 

o 

• 

G/ATOR 

• 

• 

o 

JPALS  Inc  1 A 

o 

n  KP3 

in  future 

LHA  6 

• 

■ 

LCS 

o 

• 

LCS  Packages 

o 

■ 

MQ-4C  Triton 

• 

• 

o 

Fire  Scout 

• 

• 

SSC 

• 

O 

o 

KP  2  KP  3 

in  future  in  future 


Average 
acquisition 
cycle  time 

(in  months) 


Note:  In  addition  to  research  and  development  and  procurement 
costs,  total  acquisition  cost  includes  acquisition  related  operations 
and  maintenance  and  system-specific  military  construction  costs. 


All  applicable  knowledge  practices  were  completed 

OOne  or  more  applicable  knowledge  practices  were 
not  completed 

All  knowledge  practices  were  not  applicable 
Information  not  available  for  one  or  more 
knowledge  practice 

Note:  The  DDG  51  Flight  III  Destroyer  is  excluded  from  the 
summary  above  since  we  only  present  our  observations  about 
the  program  in  a  1-page  assessment  format. 


Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 
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Navy  and  Marine  Corps  Program  Assessments 

Page 

2-page  assessments  Number 

AIM-9X  Block  II  Air-to-Air  Missile  (AIM-9X  Block  II)  87 

Air  and  Missile  Defense  Radar  (AMDR)  89 

Amphibious  Combat  Vehicle  (ACV)  91 

CH-53K  Heavy  Lift  Replacement  Helicopter  (CH-53K)  93 

DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000)  95 

Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78)  97 

Ground/Air  Task  Oriented  Radar  (G/ATOR)  99 

Joint  Precision  Approach  and  Landing  System  Increment  1A  (JPALS  Inc  1A)  101 

LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6)  103 

Littoral  Combat  Ship  (LCS)  105 

Littoral  Combat  Ship  -  Mission  Modules  (LCS  Packages)  107 

MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C  Triton)  109 

MQ-8  (Fire  Scout)  111 

Next  Generation  Jammer  Increment  1  (NGJ  Increment  1)  113 

Offensive  Anti-Surface  Warfare  Increment  1  (OASuW  Inc  1)  115 

Ship  to  Shore  Connector  Amphibious  Craft  (SSC)  1 1 7 

VH-92A  PresidentialJHelicopter  Replacement  Program  (VH-92A)  119 

1-page  assessments 

Amphibious  Ship  Replacement  (LX(R))  121 

DDG  51  Flight  III  Destroyer  122 

Fleet  Replenishment  Oiler  (T-AO(X))  123 

Ohio-Class  Replacement  (OR)  124 

P-8A  Increment  3  125 

Unmanned  Carrier-Launched  Airborne  Surveillance  and  Strike  (UCLASS)  System  126 
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Lead  Component:  Navy 


Common  Name:  AIM-9X  Block  II 


AIM-9X  Block  II  Air-to-Air  Missile  (AIM-9X  Block  II) 

The  AIM-9X  Block  II  is  a  Navy-led  program  to 
acquire  short-range  air-to-air  missiles  for  the  F-35, 
the  Navy's  F-18,  and  the  Air  Force's  F-15,  F-16,  and 
F-22A  fighter  aircraft.  It  is  designed  to  detect, 
acquire,  intercept,  and  destroy  a  range  of  airborne 
threats.  Block  II  includes  hardware  and  software 
upgrades  intended  to  improve  the  range  from  which 
the  AIM-9X  can  engage  targets,  target 
discrimination,  and  interoperability.  It  was 
designated  a  major  defense  acquisition  program  in 
2011. 


Source:  U.S.  Navy. 


Concept  System  development  Production 


▲ 
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▲ 
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▲  ▲ 
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▲ 
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Preliminary 

Critical 

Low-rate 

Operational  Initial 

Full-rate 

GAO 

start 

design  review 

design  review 

decision 

test  complete  capability 

decision 

review 

(2004) 

(3/07) 

(9/07) 

(6/11) 

(3/15)  (3/15) 

(8/15) 

(1/16) 

Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Raytheon  Missile 

As  of 

Latest 

Percent 

Systems 

12/2011 

07/2015 

change 

Program  office:  Patuxent  River,  MD 

Research  and  development  cost 

$182.4 

$550.6 

201.9% 

Funding  needed  to  complete: 

Procurement  cost 

$4,104.3 

$2,955.4 

-28.0% 

R&D:  $335.7  million 

Total  program  cost 

$4,286.7 

$3,506.0 

-18.2% 

Procurement:  $2,113.5  million 

Program  unit  cost 

$0,714 

$0,584 

-18.2% 

Total  funding:  $2,449.2  million 

Total  quantities 

6,000 

6,000 

0.0% 

Procurement  quantity:  4,432 

Acquisition  cycle  time  (months) 

39 

45 

15.4% 

The  AIM-9X  Block  II  entered  production  in  June 
2011  with  mature  critical  technologies,  a  stable 
and  demonstrated  design,  and  production 
processes  that  had  been  demonstrated  on  a 
production  line  but  were  not  in  control.  In  July 
2013,  the  Navy  suspended  operational  testing 
due  to  missile  performance  issues.  The  program 
resumed  operational  testing  in  June  2014  after 
identifying  root  causes  and  fixes  for  these  issues, 
and  completed  testing  in  March  2015.  The  system 
was  found  to  be  operationally  effective  and 
suitable  and  the  program  received  approval  to 
enter  full-rate  production  in  August  2015,  more 
than  a  year  later  than  initially  planned.  The  AIM- 
OX  Block  II  program  office  is  working  to  resolve 
additional  minor  deficiencies  with  the  missile 
identified  during  testing;  aircraft  program  offices 
are  working  to  address  integration  issues. 
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Common  Name:  AIM-9X  Block  II 


AIM-9X  Block  II  Program 

Technology  and  Design  Maturity 

AIM-9X  Block  II  entered  operational  testing  with  its 
critical  technologies  mature  and  its  design  stable 
and  demonstrated.  According  to  the  Navy's  May 
2011  technology  readiness  assessment,  Block  II 
involves  the  integration  of  mature  technologies, 
including  a  new  active  optical  target 
detector/datalink,  an  upgraded  electronics  unit,  and 
new  operational  flight  software.  The  program 
estimates  that  85  percent  of  Block  II  components 
are  unchanged  from  Block  I.  The  Navy  suspended 
operational  testing  on  the  AIM-9X  Block  II  in  July 
2013  due  to  missile  performance  deficiencies 
related  to  hardware  in  the  inertial  measurement  unit 
and  concerns  about  the  missiles  target  acquisition 
time,  which  required  a  minor  hardware  upgrade  and 
a  software  fix.  The  contractor  delivered  solutions  to 
these  issues  in  January  2014  and  the  program  re¬ 
entered  operational  testing  in  June  2014.  AIM-9X 
Block  II  competed  operational  testing  in  March  2015 
and  was  found  to  be  operationally  effective  and 
suitable.  While  issues  from  the  prior  round  of  testing 
were  resolved,  the  recent  testing  identified  four 
additional  deficiencies  related  to  reliability,  software, 
and  performance  of  the  missile  as  well  as  five  major 
deficiencies  related  to  aircraft  integration  which  are 
being  addressed  by  the  aircraft  program  offices. 
Program  officials  said  that  most  of  the  missile 
deficiencies  will  be  corrected  with  the  next  software 
update  scheduled  for  2020.  The  major  integration 
failures  will  be  corrected  in  early  2016  through  a 
software  upgrade  on  F-18  aircraft. 

Production  Maturity 

AIM-9X  Block  II  began  production  in  June  2011 ,  with 
manufacturing  processes  that  had  been 
demonstrated  on  a  pilot  production  line  but  were  not 
in  control.  Since  the  start  of  production,  the  program 
has  further  matured  its  processes,  and  program 
officials  stated  that  they  are  now  at  a  manufacturing 
readiness  level  that  indicates  they  are  in  control.  In 
the  most  recent  operational  testing,  a  production 
process  deficiency  caused  a  misfire.  Program 
officials  said  that  this  is  related  to  quality  issues  in 
the  production  line,  which  they  have  addressed 
through  additional  quality  assurance  procedures, 
and  a  potential  reliability  issue  related  to  the  ignition 
safety  device.  The  program  is  still  determining  the 
root  cause  of  this  issue. 


Other  Program  Issues 

The  suspension  of  operational  testing  delayed  the 
program's  full-rate  production  decision  from  April 
2014  to  August  2015.  Production  of  AIM-9X  Block  II 
continued  during  the  suspension  of  operational 
testing,  but  the  program  office  said  they  did  not 
accept  the  delivery  of  any  additional  missiles  until 
April  2015,  when  the  issues  were  resolved  and  the 
units  had  been  retrofitted.  In  total  18  percent  of  the 
total  quantities  were  procured  prior  to  the  full  rate 
production  and  have  been  retrofitted  to  correct  the 
issues  found  in  both  iterations  of  operational  testing. 

The  AIM-9X  Block  III  program  was  cancelled  in 
fiscal  year  2014.  However,  funding  for  several  of  the 
Block  III  planned  improvements,  including  increased 
lethality,  enhanced  infra-red  countermeasures,  and 
improved  insensitive  munitions  performance,  was 
moved  to  the  AIM-9X  Block  II  program,  as  part  of 
the  System  Improvement  Program  III  (SIP  III).  A 
$312  million  contract  for  this  effort  was  awarded  in 
September  201 5. The  contract  for  this  program  will 
also  address  component  obsolescence  and 
implement  cost  reduction  initiatives. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment, 
program  officials  stated  that  the  AIM-9X  Block  II 
missile  development  program  was  highly 
successful,  bringing  changes  that  were 
implemented  to  address  obsolescence  and 
providing  a  greatly  increased  performance  envelope 
over  the  previous  AIM-9X  missile  design.  According 
to  officials,  while  the  program  experienced  testing 
delays,  changes  to  hardware  and  software  resulted 
in  significant  performance  improvements,  verified 
with  laboratory,  ground,  flight  and  live  fire  testing. 
The  program  office  also  provided  technical 
comments,  which  were  incorporated  where  deemed 
necessary. 
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Lead  Component:  Navy 


Common  Name:  AMDR 


Air  and  Missile  Defense  Radar  (AMDR) 


The  Navy’s  AMDR  is  a  next-generation  radar 
program  designed  to  support  surface  warfare  and 
integrated  air  and  missile  defense.  AMDR  is 
developing  an  S-band  radar  for  ballistic  missile  and 
air  defense  and  a  radar  suite  controller  to  provide 
radar  resource  management,  coordination,  and 
interface  to  the  combat  system  to  support  DDG  51 
Flight  III  ships.  For  horizon  search  capability,  the 
Navy  plans  to  use  an  existing  X-band  radar  for  the 
first  12  DDG  51  Flight  III  ships,  with  a  future  X-band 
radar  program  expected  to  support  subsequent 
ships. 


Source:  U.S.  Navy. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Raytheon 

As  of 

Latest 

Percent 

Program  office:  Washington,  DC 

10/2013 

07/2015 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$1,945.0 

$1,793.0 

-7.8% 

R&D:  $551 .4  million 

Procurement  cost 

$4,022.7 

$3,408.8 

-15.3% 

Procurement:  $3,408.8  million 

Total  program  cost 

$5,997.8 

$5,231.6 

-12.8% 

Total  funding:  $3,960.2  million 

Program  unit  cost 

$272,627 

$237,801 

-12.8% 

Procurement  quantity:  22 

Total  quantities 

22 

22 

0.0% 

Acquisition  cycle  time  (months) 

155 

155 

0.0% 

AMDR  completed  critical  design  review  in  April 
2015,  with  its  four  critical  technologies 
approaching  full  maturity  and  its  design  stable. 
Two  of  four  planned  software  builds  have  been 
completed,  with  the  other  two  builds  expected  to 
be  completed  by  September  201 6.  The  contractor 
is  finalizing  a  full-scale  radar  array  prototype, 
which  is  scheduled  to  begin  land-based  testing  at 
the  Navy’s  Pacific  Missile  Range  Facility  in 
summer  201 6. The  Navy’s  plans  for  the  program 
do  not  include  testing  the  radar  system  at  sea 
until  after  it  is  installed  on  a  DDG  51  Flight  III  ship. 
The  AMDR  test  and  evaluation  master  plan  has 
not  been  approved  by  DOD’s  Director, 
Operational  Test  and  Evaluation  (DOT&E),  and 
no  approval  is  expected  until  use  of  an  unmanned 
self-defense  test  ship  is  incorporated  into 
operational  testing. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  ^ 

environment  ^ 

•  Demonstrate  all  critical  technologies  in  an  operational  r 

environment 

•  Complete  preliminary  design  review  £ 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings  0 

•  Test  a  system-level  integrated  prototype  0 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■*■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 


Page  89 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


Common  Name:  AMDR 


AMDR  Program 

Technology  and  Design  Maturity 

AMDR's  four  critical  technologies-digital-beam- 
forming,  transmit-receive  modules,  software,  and 
digital  receivers/exciters-are  nearing  full  maturity, 
and  the  program  is  expected  to  deliver  its  first  radar 
to  DDG  51  Flight  III,  as  scheduled,  in  spring  2020. 
The  contractor  completed  developmental  testing  of 
an  early  prototype  consisting  of  key  subsystems  in 
July  2012.  In  April  2015,  the  program  office 
completed  a  critical  design  review,  with  100  percent 
of  design  drawings  finalized  and  releasable.  To 
support  initial  integration  between  the  radar  and  the 
combat  system,  the  AMDR  contractor  developed 
and  delivered  an  AMDR  simulator  to  the  combat 
system  contractor  in  September  2015.  The  AMDR 
contractor  is  also  developing  a  radar  emulator- 
scheduled  to  be  completed  in  spring  2016-that  is 
intended  to  support  system  testing  and  early 
combat  system  software  integration.  The  simulator 
and  emulator  are  expected  to  help  inform  the 
program’s  knowledge  of  the  radar  and  combat 
system  interface  performance  prior  to  a  6-month 
risk  reduction  test  period  planned  for  the  second 
half  of  fiscal  year  201 7. 

Additionally,  the  contractor  has  built  and  tested  a 
second  prototype  at  its  facility,  which  is  a  single  14- 
foot  S-band  radar  array-the  final  configuration  for 
DDG  51  Flight  III  ships  will  be  a  four-faced  array.  In 
July  2016,  this  prototype  is  expected  to  be  delivered 
to  the  Navy's  Pacific  Missile  Facility  (PMF)  in  Hawaii 
for  testing  in  a  more  representative  environment. 
The  Navy  has  allotted  15  months  in  the  AMDR 
schedule  to  install  this  asset  at  the  lab  and  complete 
test  activities  prior  to  a  low-rate  initial  production 
decision  in  September  2017.  This  production 
decision  will  be  made  prior  to  combat  system 
integration  and  test,  so  any  design  issues  identified 
through  testing  will  have  to  be  addressed  during 
production. 

The  AMDR  program  includes  significant  software 
development,  which  is  being  completed  in  four 
builds.  The  software  approach  includes  upfront 
requirements  and  architecture  analysis  for  each 
build,  as  well  as  continuous  integration  of  new 
software  and  automated  testing  to  ensure 
functionality  and  performance.  The  first  two  already 
completed  developed  basic  infrastructure,  anti-air 
warfare,  and  ballistic  missile  defense  capabilities. 


The  third  and  fourth  builds  are  intended  to  provide 
the  full  extent  of  radar  capabilities,  including  debris 
detection  and  mitigation  and  advanced 
discrimination  of  missile  threats.  Build  three  is 
scheduled  to  be  completed  in  January  2016  and  the 
final  build  completion  is  planned  for  September 
2016.  The  Navy  also  has  approved  plans  to 
upgrade  the  combat  system  for  integration  with 
AMDR,  but  the  requirements  for  the  upgrade  have 
not  yet  been  defined.  The  interface  between  AMDR 
and  the  combat  management  system  will  require  a 
significant  software  development  effort,  with 
software  builds  expected  to  be  completed  in  fiscal 
year  2021. 

Other  Program  Issues 

In  201 3,  DOT&E  disapproved  AMDR's  Test  and 
Evaluation  Master  Plan  because  of  operational 
realism  concerns,  noting  that  use  of  an  unmanned 
AMDR-  and  Aegis-equipped  self-defense  test  ship 
is  needed  to  ensure  adequate  operational  testing. 
No  decision  has  been  made  on  whether  a  test  ship 
will  be  used  for  the  testing.  If  a  test  ship  is  used, 
early  DOD  estimates  suggest  that  operational 
testing  costs  will  increase  by  $300  to  $400  million. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
Navy  stated  they  intend  to  complete  the 
requirements  for  the  upgrade  of  the  combat  system 
for  integration  with  AMDR  by  the  fall  of  2016.  The 
program  also  provided  technical  comments,  which 
were  incorporated  as  appropriate. 
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Lead  Component:  Navy 


Common  Name:  ACV 


Amphibious  Combat  Vehicle  (ACV) 


The  Marine  Corps' ACV  is  the  successor  program  to 
the  canceled  Expeditionary  Fighting  Vehicle.  The 
ACV  is  intended  to  transport  Marines  from  ship  to 
shore  and  provide  armored  protection  on  land.  It  will 
potentially  replace  all  or  a  portion  of  the  existing 
Assault  Amphibious  Vehicle  fleet.  The  ACV 
acquisition  approach  calls  for  three  increments  of 
development-1.1,  1.2,  and  2.0-with  increasing 
amphibious  capability  provided  by  each  successive 
increment.  We  assessed  the  first  increment,  ACV 
1.1. 


Source:  U.S.  Marine  Corps. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  TBD 

As  of 

Latest 

Percent 

Program  office:  Stafford,  VA 

07/2015 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

N/A 

$904.0 

N/A 

R&D:  $632.8  million 

Procurement  cost 

N/A 

$1,111.8 

N/A 

Procurement:  $1,111.9  million 

Total  program  cost 

N/A 

$2,040.5 

N/A 

Total  funding:  $1,769.4  million 

Program  unit  cost 

N/A 

$8,502 

N/A 

Procurement  quantity:  204 

Total  quantities 

N/A 

240 

N/A 

Acquisition  cycle  time  (months) 

N/A 

57 

N/A 

While  the  ACV  program  considers  both  of  its 
critical  technologies  to  be  fully  mature,  one  of  the 
technologies  has  not  been  demonstrated  in  the 
ACV's  operational  environment.  The  program 
does  not  plan  to  hold  a  preliminary  design  review 
before  the  start  of  system  development  and  will 
instead  combine  it  with  the  critical  design  review 
scheduled  for  February  2016.  Eliminating 
preliminary  design  review  before  system 
development  start  may  limit  knowledge  gained 
and  increase  risk.  The  program  also  plans  to 
conduct  concurrent  testing  and  production  that 
could  leave  it  at  risk  for  design  changes  after 
production  begins,  as  well  as  costly  retrofits,  if 
deficiencies  are  found  during  testing.  The 
program  awarded  two  development  contracts  in 
November  2015;  however,  contract  performance 
has  been  delayed  by  a  bid  protest  filed  with  GAO. 


Attainment  of  Product  Knowledge 


Projected  as  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 


Demonstrate  all  critical  technologies  in  an  operational 
environment 


o 


Complete  preliminary  design  review 


Product  design  is  stable 


o 


Release  at  least  90  percent  of  design  drawings  % 

Test  a  system-level  integrated  prototype  o 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


9  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 

Not  applicable 
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Common  Name:  ACV 


ACV  Program 

Technology  Maturity 

While  the  ACV  1.1  program  considers  its  critical 
technologies  fully  mature,  one  of  the  technologies 
has  not  yet  been  demonstrated  in  the  ACV's 
operational  environment.  A  December  2014 
technology  readiness  assessment  identified  two 
critical  technology  elements  and  considered  both  to 
be  mature.  One  critical  technology-drivers  vision 
enhancement-was  considered  fully  mature  since  it 
employs  an  existing  system  used  on  the 
Amphibious  Assault  Vehicle.  The  second  critical 
technology-the  remote  weapon  station-was  also 
assessed  as  mature.  However,  this  technology  has 
not  been  demonstrated  in  the  marine  environment, 
in  which  the  ACV  is  expected  to  operate.  As  a 
result,  the  technology  could  entail  a  higher  level  of 
risk  and  may  require  additional  testing  as 
development  begins.  Development  of  future 
increments  of  ACV  will  require  new  technologies 
beyond  those  included  in  the  first  increment. 

Design  Maturity 

The  ACV  program  planned  to  hold  a  preliminary 
design  review  and  critical  design  review  about  three 
months  after  system  development  start.  As  the 
development  contracts  are  the  subject  of  a  bid 
protest,  that  event  will  likely  be  delayed.  A 
preliminary  design  review  is  typically  required 
before  development  begins.  Best  practices 
recommend  this  step  to  ensure  that  design 
requirements  and  cost  and  schedule  estimates  are 
informed  by  this  review  before  committing  to  system 
development.  Bypassing  this  review  before  system 
development  may  limit  the  knowledge  gained  and 
increase  cost  and  schedule  risk.  According  to  the 
program  office,  officials  expect  that  90  percent  of 
engineering  drawings  will  be  completed  by  critical 
design  review,  in  alignment  with  best  practices. 
However,  combining  the  preliminary  and  critical 
design  reviews  may  limit  the  time  available  to  the 
program  to  address  any  issues  identified  and 
ensure  that  sufficient  knowledge  is  attained  prior  to 
system  demonstration. 

Production  Maturity 

The  current  ACV  program  schedule  includes 
concurrency  between  testing  and  production  that 
could  increase  program  risk,  with  approximately  10 
months  of  system-level  developmental  testing 
expected  after  the  start  of  production.  According  to 


program  officials,  enough  developmental  testing, 
and  a  required  operational  assessment,  will  be 
completed  to  support  the  production  decision 
planned  for  February  2018.  Both  DOD  acquisition 
policy  and  our  prior  work  acknowledge  that  some 
degree  of  concurrency  between  initial  production 
and  developmental  testing  may  be  necessary. 
However,  our  past  work  has  also  shown  that 
beginning  production  before  demonstrating  that  a 
design  is  mature  and  will  work  as  intended 
increases  the  risk  of  discovering  deficiencies  during 
production  that  could  require  substantial  design 
changes,  costly  modifications,  and  retrofits  to 
vehicles  already  built. 

Other  Program  Issues 

The  ACV  acquisition  approach  included  competition 
through  competitive  prototyping  prior  to  the  start  of 
system  development  and  will  continue  competition 
through  development  and  production.  The  program 
awarded  development  contracts  to  two  contractors 
in  November  2015,  and  plans  to  maintain 
competition  until  the  start  of  production  for  the  ACV 
1.1  variant.  However,  contract  performance  has 
been  delayed  pending  the  resolution  of  issues 
raised  in  a  bid  protest  filed  with  GAO. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment, 
program  officials  noted  that  the  latest  cost  data 
reported  in  this  assessment  are  based  on  a 
program  manager's  estimate  and  cannot  be 
finalized  until  the  ACV's  acquisition  program 
baseline-the  first  full  cost  and  schedule  estimates 
generally  established  at  system  development  start- 
is  approved. 
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Lead  Component:  Navy 


Common  Name:  CH-53K 


CH-53K  Heavy  Lift  Replacement  Helicopter  (CH-53K) 


The  Marine  Corps'  CH-53K  heavy-lift  helicopter  is 
intended  to  transport  armored  vehicles,  equipment, 
and  personnel  to  support  operations  deep  inland 
from  a  sea-based  center  of  operations.  The  CH-53K 
is  expected  to  replace  the  legacy  CH-53E  helicopter 
and  provide  increased  range  and  payload, 
survivability  and  force  protection,  reliability  and 
maintainability,  and  coordination  with  other  assets, 
while  reducing  total  ownership  costs. 


Source:  Sikorsky  Aircraft  Corporation. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Sikorsky  Aircraft 

As  of 

Latest 

Percent 

Corporation 

12/2005 

07/2015 

change 

Program  office:  Patuxent  River,  MD 

Research  and  development  cost 

$4,698.2 

$6,755.0 

43.8% 

Funding  needed  to  complete: 

Procurement  cost 

$13,066.4 

$18,935.6 

44.9% 

R&D:  $1,581.8  million 

Total  program  cost 

$17,764.6 

$25,708.2 

44.7% 

Procurement:  $18,935.6  million 

Program  unit  cost 

$113,876 

$128,541 

12.9% 

Total  funding:  $20,521.9  million 

Total  quantities 

156 

200 

28.2% 

Procurement  quantity:  194 

Acquisition  cycle  time  (months) 

119 

163 

37.0% 

The  CH-53K  conducted  its  first  flight  in  October 
2015,  and  the  program's  two  critical  technologies 
-housed  within  the  main  rotor  blade  and  main 
gearbox-have  begun  testing  in  an  operational 
environment.  While  the  program  completed  its 
critical  design  review  more  than  5  years  ago, 
unexpected  redesigns  to  critical  components 
have  delayed  aircraft  assembly  and  testing  and 
have  slowed  delivery  of  test  aircraft.  As  a  result  of 
these  issues,  the  program's  low-rate  initial 
production  decision  has  been  delayed  again  to 
February  2017-approximately  8  additional 
months  since  our  last  assessment.  To  date,  two  of 
the  five  test  aircraft  have  been  delivered,  the  third 
is  complete  but  has  not  yet  begun  flight  testing, 
and  the  remaining  two  have  been  delayed  due  to 
unexpected  redesigns.  Achieving  initial  capability 
is  now  expected  nearly  4  years  later  than  planned 
due  to  these  challenges. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  _ 

environment  W 

•  Demonstrate  all  critical  technologies  in  an  operational  ^ 

environment  ” 

•  Complete  preliminary  design  review  0 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings  ■  ■  ■  ■ 

•  Test  a  system-level  integrated  prototype  0 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


0  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  CH-53K 


CH-53K  Program 

Technology  and  Design  Maturity 

The  CH-53K  conducted  its  first  flight  in  October 
2015-nearly  3  years  later  than  originally  planned 
and  has  logged  a  number  of  test  flights  to  date.  As  a 
result  of  this  testing,  the  program's  two  critical 
technologies-housed  within  the  main  rotor  blade 
and  main  gearbox-have  begun  testing  in  an 
operational  environment.  Reaching  this  point  more 
than  5  years  after  the  program's  critical  design 
review  and  nearly  10  years  after  system 
development  start  is  not  consistent  with  best 
practices. 

Unanticipated  design  changes  to  non-critical 
technology  components  after  critical  design  review 
continue  to  cause  flight  test  and  production  delays. 
For  example,  components  within  the  rear  module 
assembly,  part  of  the  main  gear  box,  required  a 
number  of  redesigns.  Most  recently,  equipment  that 
holds  one  of  the  gears  in  place  failed,  which 
required  a  reconfiguration  of  the  rear  module 
assembly  retention  design.  The  new  design  has 
been  completed  and  installed  on  the  ground  test 
vehicle  and  the  second  flight  test  aircraft,  which  is 
expected  to  achieve  its  initial  flight  in  early  2016. 
The  redesigned  gear  box  will  also  be  installed  in  the 
next  two  flight  test  aircraft,  but  has  not  yet  been 
installed  in  the  current  flight  test  aircraft.  The  current 
flight  test  aircraft  uses  the  prior  design  and  will  be 
held  to  a  flight  life  limitation.  Schedule  delays  also 
occurred  due  to  late  component  deliveries  and 
qualification  issues  for  the  main  rotor  blades,  main 
rotor  head,  and  tail  rotor  blades.  The  program 
continues  to  monitor  and  mitigate  these  delays 
when  possible.  The  failure  to  demonstrate 
technology  and  design  maturity  at  appropriate 
points  earlier  in  system  development  is  one  reason 
why  the  CH-53K  program  now  expects  to  reach  its 
initial  operational  capability  in  late  2019,  nearly  4 
years  later  than  originally  planned. 

Production  Maturity 

To  date  the  CH-53K  program  has  taken  delivery  of 
the  ground  test  vehicle  with  the  redesigned  main 
gear  box,  and  the  first  flight  test  aircraft.  The  second 
flight  test  aircraft  is  complete  and  has  undergone 
installation  of  the  redesigned  main  gear  box  but  has 
not  yet  begun  flight  tests.  The  unexpected 
redesigns  of  the  aircraft's  various  gear  boxes  as 
well  as  the  late  delivery  of  some  components  have 


delayed  delivery  of  the  remaining  two  engineering 
design  model  test  aircraft.  This  has  created  delays 
at  the  production  facility  where  parts  are  received 
from  vendors,  which  is  expected  to  impact  the  flight 
test  schedule. 

As  a  result  of  these  delays,  the  program's  low-rate 
initial  production  decision  has  been  moved  from 
June  2016  to  February  2017-an  eight  month  delay 
since  our  last  assessment.  Delays  to  this  production 
decision  resulted  in  a  program  deviation  report 
which  was  reviewed  by  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics, 
who  approved  the  schedule  extension  and  decided 
not  to  require  the  program  to  establish  a  new 
acquisition  program  baseline  until  the  CH-53K's 
low-rate  production  decision  is  reached.  According 
to  program  officials,  the  expected  date  for  the 
production  decision  is  February  2017.  The  current 
acquisition  program  baseline  will  remain  unchanged 
until  the  production  decision  occurs. 

Program  Office  Comments 

The  program  office  provided  technical  comments, 
which  were  incorporated  as  appropriate. 
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Lead  Component:  Navy 


Common  Name:  DDG  1000 


DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 


The  Navy's  DDG  1000  destroyer  is  a  multimission 
surface  ship  designed  to  provide  advanced 
capability  for  littoral  operations  and  land-attack  in 
support  of  forces  ashore.  The  ship  will  feature  an 
integrated  power  system,  a  total  ship  computing 
environment,  and  an  advanced  gun  system.  Lead 
ship  delivery-comprised  of  the  ship's  hull, 
mechanical,  and  electrical  systems-is  expected 
April  2016,  an  18-month  delay.  Construction  is 
underway  on  the  remaining  two  ships  in  the  class. 


Source:  General  Dynamics  -  Bath  Iron  Works. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  BAE  Systems,  Bath 

As  of 

Latest 

Percent 

Iron  Works 

01/1998 

12/2014 

change 

Program  office:  Washington,  DC 

Research  and  development  cost 

$2,444.3 

$10,751.6 

339.9% 

Funding  needed  to  complete: 

Procurement  cost 

$34,896.9 

$12,314.5 

-64.7% 

R&D:  $133.3  million 

Total  program  cost 

$37,341.1 

$23,066.1 

-38.2% 

Procurement:  $847.2  million 

Program  unit  cost 

$1,166.91 

$7,688,712 

559.9% 

Total  funding:  $980.4  million 

Total  quantities 

32 

3 

-90.6% 

Procurement  quantity:  0 

Acquisition  cycle  time  (months) 

128 

248 

93.8% 

The  DDG  1000  program  experienced  significant 
challenges  with  activating  and  testing  the  lead 
ship's  hull,  mechanical,  and  electrical  systems 
resulting  in  delays  to  lead  ship  delivery.  These 
delays  could  also  affect  the  start  of  mission 
system  activation  and  verification  that  the  ship 
can  meet  performance  requirements.  Delivery  of 
the  lead  ship's  hull,  mechanical,  and  electrical 
systems  is  now  expected  in  April  2016,  an  18- 
month  delay.  Program  officials  said  delays  were 
initially  driven  by  production  challenges, 
particularly  a  shortage  of  labor  to  complete 
electrical  work.  The  program  also  encountered 
significant  technical  issues  related  to  the 
integrated  power  system,  a  critical  technology,  but 
the  program  believes  that  the  system  was 
successfully  demonstrated  during  initial  sea  trials 
in  December  2015. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  ^ 

environment  ™ 

•  Demonstrate  all  critical  technologies  in  an  operational 
environment 

•  Complete  preliminary  design  review  £ 


Product  design  is  stable 


•  Complete  three-dimensional  product  model  0 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  DDG  1000 


DDG  1000  Program 

Technology  Maturity 

While  3  of  the  DDG  1000's  11  critical  technologies 
are  fully  mature,  the  remaining  8  will  not  be 
demonstrated  in  a  realistic  environment  until  testing 
aboard  the  lead  ship.  Reliability  of  the  ship's  power 
conversion  and  distribution  system-a  key  element 
of  the  integrated  power  system-was  the  top 
technical  risk  for  completion  of  the  sea  trials 
required  prior  to  delivery.  The  program  believes  that 
the  integrated  power  system  was  successfully 
demonstrated  during  initial  sea  trials  in  December 

2015.  Issues  with  the  power  conversion  and 
distribution  system  were  due,  in  part,  to  the  Navy's 
decision  to  not  fully  test  and  validate  the 
performance  of  the  system  in  a  representative 
environment  prior  to  installation  on  the  ship.  The 
Navy  conducted  land-based  testing  of  the 
integrated  power  system  with  a  configuration 
representing  only  half  of  the  system,  with  plans  to 
install  the  test  equipment  on  DDG  1002,  which  it 
believed  was  the  cost  effective  approach  to  risk 
reduction.  As  a  result,  the  Navy  reported  that 
numerous  issues  with  the  power  conversion  and 
distribution  system  were  discovered  during  land- 
based  testing  but  not  resolved  before  ship 
installation.  Because  land-based  testing  did  not 
include  testing  to  replicate  shipboard  power  loads, 
power  disturbances  were  discovered  during 
shipboard  testing. 

The  program  reported  that  all  mission  system 
equipment  for  the  first  two  ships  has  been  delivered 
and  installed,  with  mission  system  activation 
planned  after  lead  ship  delivery.  The  contract  for  the 
third  ship's  mission  system  equipment  was  awarded 
in  December  2015.  Testing  of  modifications  to  the 
multifunction  radar  to  include  a  volume  search 
capability  is  ongoing.  Multiple  tracking  exercises 
with  the  multifunction  radar  were  conducted  at 
Wallops  Island  in  2015.  A  follow-on  tracking 
exercise  is  planned  in  June  2016.  According  to 
program  officials,  software  acceptance  testing  for 
the  eighth  and  final  software  build  for  the  total  ship 
computing  environment  has  been  completed  and 
will  be  accepted  in  January  2016.  Low-rate  initial 
production  decision  on  the  long  range  land  attack 
projectile  is  planned  for  second  quarter  fiscal  year 

2016. 


Design  and  Production  Maturity 

The  DDG  1000  design  is  mature,  but  ongoing 
development  and  shipboard  testing  of  technologies 
may  result  in  design  changes.  As  of  December 
2015,  the  program  reported  that  construction  of  the 
three  ships  in  the  class  was  98,  84,  and  43  percent 
complete,  respectively.  Program  officials  said 
delays  to  lead  ship  delivery  were  initially  driven  by 
challenges  in  completing  electrical  work,  with  the 
shipbuilder  citing  resource  shortages  and  workforce 
turnover.  Significant  technical  issues  with  activation 
and  testing  of  the  lead  ship's  hull,  mechanical,  and 
electrical  systems  have  further  delayed  lead  ship 
delivery.  As  of  December  2015,  the  program 
reported  that  activation  of  the  lead  ship's  hull, 
mechanical  and  electrical  systems  was  83  percent 
complete.  The  program  also  reported  that  initial  sea 
trials  in  December  2015  demonstrated  several  ship 
systems  including  small  boat  operations,  anchoring, 
integrated  power  system,  and  auxiliary  systems  and 
that  primary  risk  reduction  objectives  were 
successfully  met.  Lead  ship  delivery  is  expected  in 
April  2016. 

Other  Program  Issues 

Late  delivery  of  the  lead  ship's  hull,  mechanical,  and 
electrical  systems  delayed  the  start  of  mission 
system  activation  and  verification  that  the  ship  can 
meet  performance  requirements.  The  current 
estimate  for  initial  operating  capability  is  September 
2018,  almost  two  years  later  than  planned.  A 
revision  of  the  program's  acquisition  program 
baseline  to  account  for  these  schedule  delays  has 
not  yet  been  approved. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
Navy  concurred  with  our  findings  and  provided 
additional  information.  The  Navy  noted  that  the 
program  has  made  significant  progress  in  the  test 
and  activation  phase  as  several  ship  systems  were 
demonstrated  during  initial  sea  trials  in  December 
2015.  The  Navy  noted  that  analysis  continues  and 
any  identified  corrective  actions  will  be  prioritized  to 
best  support  the  continuing  schedule  of  test  and  trial 
events.  The  Navy  added  that  it  continues  to  focus 
on  delivery  of  the  lead  ship's  hull,  mechanical,  and 
electrical  systems;  sail  away  and  planning  mission 
systems  activation.  The  Navy  also  provided 
technical  comments,  which  were  incorporated 
where  appropriate. 
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Lead  Component:  Navy 


Common  Name:  CVN  78 


Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78) 

The  Navy  developed  the  Ford-class  nuclear- 
powered  aircraft  carrier  to  introduce  new  propulsion, 
aircraft  launch  and  recovery,  and  survivability 
capabilities  to  the  carrier  fleet.  The  Ford-class  is  the 
successor  to  the  Nimitz-class  aircraft  carrier,  and  its 
new  technologies  are  intended  to  create  operational 
efficiencies  while  enabling  a  25  percent  increase  in 
operational  aircraft  flights  as  compared  to  legacy 
carriers.  The  Navy  also  expects  the  new 
technologies  to  enable  Ford-class  carriers  to 
operate  with  reduced  manpower. 


Source:  U.S.  Navy. 
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Program  Essentials 

Prime  contractor:  Huntington  Ingalls 
Industries 

Program  office:  Washington,  DC 
Funding  needed  to  complete: 

R&D:  $378.1  million 
Procurement:  $14,310.7  million 
Total  funding:  $14,772.3  million 
Procurement  quantity:  1 


Program  Performance  (fiscal  year  2016  dollars  in  millions) 


Research  and  development  cost 

As  of 
04/2004 

$5,154.0 

Latest 

07/2015 

$5,059.1 

Percent 

change 

-1.8% 

Procurement  cost 

$33,017.6 

$31,339.1 

-5.1% 

Total  program  cost 

$38,171.6 

$36,547.9 

-4.3% 

Program  unit  cost 

$12,723,855 

$12,182,648 

-4.3% 

Total  quantities 

3 

3 

0.0% 

Acquisition  cycle  time  (months) 

137 

155 

13.1% 

The  Navy  reported  9  of  CVN  78's  13  critical 
technologies  as  mature,  though  testing  of 
immature  technologies  continues  to  reveal  issues 
and  ship  delivery  is  delayed.  CVN  78  is  over  95 
percent  complete,  but  to  manage  risks  the  Navy 
deferred  some  work  until  after  ship  delivery.  This 
could  obscure  costs  and  result  in  delivery  of  an 
incomplete  ship.  Lead  ship  procurement  costs 
grew  by  almost  23  percent  from  $10.5  billion  to 
$12.9  billion-the  limit  of  the  current  legislated  cost 
cap.  The  National  Defense  Authorization  Act  for 
Fiscal  Year  2016  reduced  the  CVN  79  cost  cap  to 
$11.4  billion.  The  Navy  adopted  a  two-phased 
acquisition  approach  for  CVN  79  and  deferred 
installation  of  some  upgrades;  however,  this 
strategy  will  delay  some  construction  and  costs  to 
after  ship  delivery. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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CVN  78  Program 

Technology  and  Design  Maturity 

The  Navy  reported  that  9  of  CVN  78's  13  critical 
technologies  are  mature,  though  testing  of 
immature  technologies  continues  to  reveal  issues. 
The  Navy  began  deadload  testing  of  the 
electromagnetic  aircraft  launch  system  (EMALS) 
from  the  ship's  deck  but  halted  testing  in  2015  due 
to  a  system  component  failure.  The  Navy  plans  to 
begin  testing  EMALS  and  the  advanced  arresting 
gear  (AAG)  on  board  CVN  78  with  aircraft  in  2016. 
AAG  began  shipboard  testing  in  July  2015,  with 
projected  completion  after  CVN  78  delivers.  The 
dual  band  radar  (DBR)  also  began  shipboard  testing 
this  year,  despite  problems  land-based  testing 
revealed  that  could  affect  the  radar's  air  traffic 
control  functionality.  Both  AAG  and  DBR  are  still 
engaged  in  land-based  testing.  The  Navy  will 
replace  DBR  on  CVNs  79  and  80  with  the 
Enterprise  Air  Surveillance  Radar  suite,  but  has  not 
yet  awarded  a  contract  to  develop  the  new  radar.  If 
the  new  radar  cannot  fit  within  the  existing  design, 
CVNs  79  and  80,  which  use  the  CVN  78  design, 
would  require  design  modifications. 

Production  Maturity 

CVN  78  is  over  95  percent  complete  and  scheduled 
to  deliver  in  May  2016,  at  the  earliest,  about  6  to  8 
weeks  later  than  planned  due  to  delayed  sea  trials. 
To  manage  remaining  risks,  the  Navy  deferred 
some  work  until  after  ship  delivery,  a  decision  that 
could  obscure  costs  and  result  in  delivery  of  an 
incomplete  ship.  Construction  continues  on  CVN 
79,  which  is  14  percent  complete.  The  Navy 
awarded  the  detail  design  and  construction  contract 
for  this  ship  in  June  2015.  In  February  2015,  the 
Navy  also  requested  the  first  year  of  advance 
procurement  funding  for  CVN  80. 

Other  Program  Issues 

In  2007,  Congress  established  a  procurement  cost 
cap  of  $1 0.5  billion  for  CVN  78  and  since  then,  lead 
ship  procurement  costs  increased  by  almost  23 
percent  to  the  current  statutory  cost  cap  of  $12.9 
billion.  The  National  Defense  Authorization  Act 
(NDAA)  for  Fiscal  Year  2016  reduced  the  cap  for 
CVN  79  to  $11.4  billion,  though  costs  for  this  ship 
may  also  increase.  The  Office  of  the  Secretary  of 
Defense  and  the  Congressional  Budget  Office 
expect  CVN  79  to  surpass  the  earlier  statutory  cost 
cap  of  $11.5  billion  by  at  least  $235  million.  The 
Navy  asserts  it  will  meet  CVN  79's  cost  cap,  but 
assumes  unprecedented  efficiency  gains  in 


construction-that  CVN  79's  production  hours  will  be 
18  percent  lower  than  CVN  78.  The  Navy  also 
adopted  a  two-phased  acquisition  approach  for 
CVN  79  that  will  shift  some  construction  to  a  post¬ 
delivery  period.  According  to  program  officials,  this 
will  enable  the  Navy  to  procure  and  install  electronic 
systems  at  the  latest  possible  date  to  prevent 
obsolescence  prior  to  ship  deployment.  However, 
this  strategy  results  in  a  less  capable  and  complete 
ship  at  delivery.  The  Navy  is  also  transferring  the 
costs  of  a  number  of  known  capability  upgrades 
from  CVN  79  to  other  accounts  by  deferring  work  to 
future  maintenance  periods-obscuring  CVN  79's 
actual  costs. 

In  August  2015,  the  Deputy  Secretary  of  Defense 
directed  the  Navy  to  complete  the  full  ship  shock 
trial  on  CVN  78,  not  CVN  79.  The  NDAA  for  fiscal 
year  2016  restricts  the  obligation  or  expenditure  of 
fiscal  year  2016  funds  for  CVN  79  until  the  Navy 
takes  certain  steps,  including  either  certifying  it  will 
conduct  the  shock  trial  on  CVN  78  before  the  ship's 
first  deployment  or  submitting  a  notification  of  the 
waiver  to  this  requirement. 

Program  Office  Comments 

In  its  comments,  the  Navy  stated  that  all  costs  to 
complete  CVN  78  will  be  included  under  the  $12.9 
billion  cost  cap.  In  2013,  the  Navy  deferred  some 
non-critical  work  to  a  post-delivery  period  to  allow 
the  shipbuilder  to  focus  on  the  completion  of  new 
technologies  and  other  critical  work  to  deliver  the 
ship  in  the  most  cost  effective  manner.  All  work  will 
be  completed  under  the  cost  cap  prior  to  the  start  of 
Initial  Operational  Testing  and  Evaluation.  The  Navy 
noted  that  the  statement  for  CVN  79,  "this  strategy 
will  delay  some  construction  and  costs  to  after  ship 
delivery",  is  incorrect.  The  Navy  will  deliver  a 
complete  and  deployable  ship  at  the  end  of  Phase  II 
construction. 

GAO  Response 

As  we  reported  in  October  2015,  CVN  78  must 
complete  its  final,  more  complex,  construction 
phase  concurrent  with  key  test  events,  with  no 
margin  for  the  unexpected.  Additional  costs  are 
likely.  The  Navy  believes  that  our  statement 
regarding  CVN  79's  capability  at  ship  delivery  is 
incorrect.  However,  as  the  Navy  states  in  its 
comments,  the  ship  will  not  be  fully  complete  and 
deployable  at  delivery  (known  as  Phase  I)  in  fiscal 
year  2022,  but  rather  much  later-in  fiscal  year  2025 
at  the  end  of  Phase  II  construction. 
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Ground/Air  Task  Oriented  Radar  (G/ATOR) 


The  Marine  Corps'  Ground/Air  Task  Oriented  Radar 
(G/ATOR)  is  an  active  electronic  scanned  array, 
three-dimensional,  short-to-medium  range,  multi¬ 
role  radar  designed  to  detect,  identify,  and  track 
threats  such  as  cruise  missiles,  rockets,  and 
artillery.  It  will  replace  five  radars.  It  is  being 
acquired  in  blocks;  later  blocks  are  mostly  software 
upgrades.  GAO  assessed  Block  1,  which  has  an  air 
defense  and  surveillance  role,  and  made 
observations  on  Block  2,  which  will  determine 
enemy  firing  positions  and  point  of  impact  for 
incoming  fire. 


Source:  U.S.  Marine  Corps. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

As  of 

Latest 

Percent 

Program  office:  Quantico,  VA 

08/2005 

07/2015 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$379.9 

$1,048.6 

176.0% 

R&D:  $238.7  million 

Procurement  cost 

$1,192.8 

$1,723.1 

44.5% 

Procurement:  $1,445.8  million 

Total  program  cost 

$1,572.7 

$2,775.4 

76.5% 

Total  funding:  $1,688.2  million 

Program  unit  cost 

$24,573 

$61,676 

151.0% 

Procurement  quantity:  39 

Total  quantities 

64 

45 

-29.7% 

Acquisition  cycle  time  (months) 

66 

138 

109.1% 

The  cost  data  includes  G/ATOR  Blocks  1  and  2 

The  acquisition  cycle  time  was  calculated  for  Block  1 

only. 

The  G/ATOR  program  received  approval  to  enter 
production  in  March  2014  with  mature 
technologies,  a  design  refined  for  production,  and 
production  processes  that  had  been 
demonstrated,  but  were  not  in  control.  In  early 
testing,  software  stability  was  a  major  reliability 
driver.  In  October  2014,  an  expert  panel  found 
G/ATOR's  reliability  requirements  did  not  reflect 
operational  needs  and  recommended  changes.  In 
March  2015,  the  Marine  Corps  revised  the 
reliability  requirements.  The  program  continues  to 
address  software  issues  during  testing.  Program 
officials  noted  G/ATOR  will  transition  to  a  new 
semiconductor  technology  in  2016,  which  will  be 
used  in  initial  operational  testing.  A  revised  test 
plan  capturing  the  new  technology  and  reliability 
requirements  is  under  final  review  by  the  Navy. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  _ 

environment  w 

•  Demonstrate  all  critical  technologies  in  an  operational  ^ 

environment  ^ 

•  Complete  preliminary  design  review  % 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

• 

•  Test  a  system-level  integrated  prototype 

• 

1  Manufacturing  processes  are  mature 

•  Demonstrate  critical  processes  are  in  control 

o 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

•  Test  a  production-representative  prototype 

• 

9  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 
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G/ATOR  Program 

Technology  and  Design  Maturity 

As  of  July  2015,  G/ATOR  reported  that  all  six  of  its 
critical  technologies  are  fully  mature,  and  that  the 
design  had  been  refined  for  production.  The 
program's  design  was  stable  at  critical  design 
review  in  2009,  but  the  number  of  total  estimated 
drawings  has  increased  as  the  program  developed 
more  detailed  drawings  to  facilitate  production.  As 
of  July  2015,  97  percent  of  the  refined  sets  of 
drawings  have  been  released.  While  G/ATOR 
hardware  has  proven  to  be  reliable,  the  program 
has  experienced  software  maturity  and  quality 
problems,  which  have  affected  the  reliability  of  the 
overall  system.  In  June  2014,  the  Navy  convened  a 
panel  of  experts  that  conducted  an  in-depth  analysis 
of  the  program,  after  it  failed  to  meet  its  reliability 
target  during  developmental  testing.  The  panel 
found  G/ATOR's  reliability  requirement  was  likely 
unachievable  and  did  not  reflect  operational  needs. 
In  response,  in  March  2015,  the  Marine  Corps 
clarified  the  requirement  to  ensure  it  was  consistent 
with  the  G/ATOR's  operational  mission  profile. 
According  to  the  program  manager,  the  program 
continues  to  make  software  updates  to  address 
quality  and  reliability  issues. 

The  program  plans  to  upgrade  the  radar's  transmit 
and  receive  modules  by  using  a  newer 
semiconductor  technology  starting  with  radars 
produced  in  2016.  According  to  program  officials, 
this  new  gallium  nitride  technology  is  mature  and 
the  fit  of  the  new  modules  will  be  the  same  as  the 
older  gallium  arsenide  modules,  which  minimizes 
design  changes.  The  gallium  nitride  technology  is 
expected  to  achieve  better  performance  with  higher 
reliability  at  a  lower  cost  by  reducing  the  number  of 
modules  required. 

Production  Maturity 

The  G/ATOR  program  received  approval  to  enter 
low-rate  production  in  March  2014  with  production 
processes  that  had  been  demonstrated  on  a 
production  line,  but  were  not  demonstrated  as  in 
control.  While  the  program's  manufacturing 
readiness  reached  the  level  recommended  by  DOD 
guidance,  it  did  not  reach  the  level  of  control  called 
for  in  GAO's  best  practices.  This  status  is 
unchanged. 


In  April  2015,  the  program  office  received  approval 
to  increase  initial  production  quantities  from  8  to  14 
radar  systems.  The  program  office  confirmed  that  of 
the  14  initial  production  radar  systems,  the  first  six 
will  use  the  older  gallium  arsenide  modules  and  the 
later  eight  will  use  the  newer  gallium  nitride 
modules.  According  to  the  program  office,  delivery 
of  the  first  production  radar  with  the  older  module 
technology  is  expected  in  early  2017. 

Other  Program  Issues 

The  G/ATOR  program  is  in  the  process  of  revising 
its  test  strategy  to  address  reliability  concerns 
raised  by  an  expert  panel,  to  include  a  test  strategy 
for  gallium  nitride,  and  clarify  the  operational 
reliability  requirements.  The  program  originally 
planned  to  conduct  operational  testing  with  the 
gallium  arsenide  configuration,  but  DOD's  Director, 
Operational  Test  and  Evaluation  raised  concerns 
about  the  production  representativeness  of  this 
configuration  as  a  majority  of  the  planned  G/ATOR 
procurements  are  with  the  newer  gallium  nitride 
modules.  Program  officials  said  testing  using 
gallium  nitride  will  occur  in  2018.  A  revised  test  plan 
capturing  the  new  technology  and  reliability 
requirements  has  been  coordinated  with 
stakeholders,  including  DOD  test  officials,  and  is 
under  final  review  by  the  Navy. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  noted  that  the  G/ATOR  program  has 
remained  on  schedule  and  decreased  the  estimated 
total  program  cost  by  more  than15  percent  since  the 
program  was  rebaselined  in  2010.  It  also  noted  that 
15  months  into  the  initial  low-rate  production 
contract  awarded  in  October  2014  for  Block  1,  the 
program  remains  on  schedule  demonstrating  that 
production  processes  are  in  control.  The  program 
office  noted  that  transition  to  the  newer  gallium 
nitride  semiconductor  technology  in  2016  is  also  on 
schedule.  It  noted  that  the  Block  2  software  contract 
was  awarded  in  August  2015,  and  that  the  program 
continues  to  refine  the  quality  of  the  system 
software  to  be  used  in  initial  low-rate  production 
hardware  deliveries  in  2017.  Technical  comments 
were  provided  by  the  program  office,  which  were 
incorporated  where  deemed  appropriate. 
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Joint  Precision  Approach  and  Landing  System  Increment  1A  (JPALS  Inc  1A) 


JPALS  Increment  1A  is  a  Navy-led  program  to 
develop  a  GPS-based  landing  system  for  aircraft 
carriers  and  amphibious  assault  ships  to  support 
operations  with  the  Joint  Strike  Fighter  and 
Unmanned  Carrier-Launched  Airborne  Surveillance 
and  Strike  System.  The  program  intends  to  provide 
reliable  precision  approach  and  landing  capability  in 
adverse  environmental  conditions.  We  assessed 
increment  1  A.  As  a  result  of  restructuring, 
previously  planned  additional  increments  are  no 
longer  part  of  the  program. 


Source:  U.S.  Navy. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Raytheon 

As  of 

Latest 

Percent 

Program  office:  Lexington  Park,  MD 

07/2008 

07/2015 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$849.8 

$1,106.3 

30.2% 

R&D:  $188.9  million 

Procurement  cost 

$228.8 

$476.8 

108.4% 

Procurement:  $476.8  million 

Total  program  cost 

$1,086.0 

$1,590.5 

46.5% 

Total  funding:  $665.7  million 

Program  unit  cost 

$29,352 

$58,908 

100.7% 

Procurement  quantity:  17 

Total  quantities 

37 

27 

-27.0% 

Acquisition  cycle  time  (months) 

75 

146 

94.7% 

JPALS  Increment  1A  began  development  in  July 
2008.  Both  of  the  program's  previously  identified 
critical  technologies  were  demonstrated  in  a 
realistic  environment  during  sea-based  flight 
testing  in  2013.  Due  to  affordability  concerns  and 
other  military  and  civilian  plans  for  similar  landing 
systems,  the  Navy  changed  the  program  in  2014 
to  accelerate  auto-landing  capabilities  and 
eliminate  other  capabilities.  Doing  so  reduced 
planned  quantities  while  increasing  development 
costs  and  resulted  in  a  critical  Nunn-McCurdy  unit 
cost  breach.  As  a  result,  the  program  is 
restructuring  and  development  will  restart  in  June 
2016.  The  program  is  currently  conducting  new 
system-level  reviews,  developing  software  and  a 
new  cost  and  schedule  baseline,  and  will 
reassess  technology  and  design  maturity. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 


•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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JPALS  Inc  1A  Program 

Technology,  Design,  and  Production  Maturity 

In  June  2014,  the  JPALS  program  was  changed  to 
accelerate  the  development  of  aircraft  auto-land 
capabilities.  The  program's  technology  and  design 
maturity  will  need  to  be  reassessed  to  account  for 
this  change.  The  program  reported  conducting  a 
system  requirements  review  in  March  2015,  and 
plans  to  conduct  a  system-level  preliminary  design 
review  in  February  2016.  This  will  be  done  in 
advance  of  the  development  restart  for  the 
restructured  program,  scheduled  for  June  2016. 
JPALS  Increment  lA's  functionality  is  primarily 
software-based  and  the  changes  require  additional 
software  development.  The  planned  completion  of 
the  software  development  efforts  necessary  to 
achieve  baseline  capabilities  has  not  yet  been 
determined.  The  current  number  of  total  drawings 
expected  for  the  restructured  program  also  has  not 
yet  been  determined.  Program  officials  stated  that 
the  Navy  conducted  an  affordability  analysis  in  June 
201 5,  and  also  stated  that  they  are  refining  cost  and 
schedule  estimates.  They  noted  that  the  requisite 
mandatory  program  certifications  will  be  completed 
prior  to  the  restructured  development  restart.  The 
program  then  plans  to  restart  system-level 
developmental  testing  in  fiscal  year  2017. 

Prior  to  this  restructuring,  the  program  had 
completed  a  number  of  activities  to  mature  its 
technology  and  design.  JPALS  Increment  1A  began 
its  original  development  in  July  2008,  held  a  critical 
design  review  in  December  2010,  and  released  all 
of  its  expected  design  drawings  at  that  time.  The 
program  began  testing  a  system-level  prototype  in 
July  2012.  Testing  of  the  system  in  its  prior 
configuration  began  in  December  2012.  The 
program  demonstrated  its  two  identified  critical 
technologies  in  a  realistic  environment  through  sea- 
based  flight  testing  and  completed  70  ship-based 
auto-landing  approaches  using  legacy  aircraft  as  of 
November  2013.  According  to  JPALS  officials,  the 
Increment  1A  program  did  not  identify  any  critical 
manufacturing  processes  due  to  the  system's 
hardware  being  comprised  primarily  of  off-the-shelf 
components.  The  program  accepted  delivery  of 
eight  engineering  development  models,  seven  of 
which  were  considered  production-representative. 


Other  Program  Issues 

In  2013,  the  Navy  conducted  a  review  of  its 
precision  approach  and  landing  capabilities  to 
address  budget  constraints  and  affordability 
concerns.  In  light  of  these  concerns,  as  well  as 
other  military  service  and  civilian  plans  to  continue 
use  of  current  landing  systems,  the  Navy 
restructured  the  JPALS  program.  The  program  was 
reduced  from  seven  increments  to  one  intended  to 
support  the  Joint  Strike  Fighter  and  Unmanned 
Carrier-Launched  Airborne  Surveillance  and  Strike 
System.  The  Navy  also  accelerated  the  integration 
of  auto-land  capabilities  originally  intended  for  the 
future  increments,  and  eliminated  both  the 
integration  of  JPALS  with  other  sea-based  legacy 
aircraft  and  the  land-based  version  of  the  system. 
These  changes  increased  the  development  funding 
required  for  auto-land  capabilities  and  reduced 
system  quantities,  resulting  in  unit  cost  growth  and 
a  critical  Nunn-McCurdy  unit  cost  breach  reported  in 
March  2014.  The  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  certified  the 
restructured  program  and  directed  the  Navy  to 
continue  risk  reduction  efforts  to  incorporate  the 
auto-land  capabilities. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  noted  that  JPALS  is  no  longer  an 
incremental  program  and  has  completed  the 
certification  process  triggered  by  the  Nunn  McCurdy 
breach.  The  requirements  originally  associated  with 
JPALS  Increments  1A  (ship's  system),  3  (manned 
auto-land  capability)  and  4  (unmanned  auto-land 
capability)  have  been  combined  into  a  single 
program.  While  the  Navy  continues  JPALS  auto¬ 
land  development  to  support  manned  and 
unmanned  aircraft,  the  engineering  development 
models  developed  and  built  under  the  original 
JPALS  Increment  1A  program  are  being  utilized  to 
support  Joint  Strike  Fighter  test  and  operational 
support  requirements. 
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Lead  Component:  Navy 


Common  Name:  LHA  6 


LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6) 

The  Navy's  LHA  6  class  will  replace  the  LHA  1 
Tarawa-class  amphibious  assault  ships.  The  LHA  6 
class  is  based  on  the  fielded  LHD  8  and  consists  of 
three  ships.  The  ships  feature  enhanced  aviation 
capabilities  and  are  designed  to  support  Marine 
Corps  assets  in  an  expeditionary  strike  group.  LHA 
6  construction  began  in  December  2008  with 
delivery  in  April  2014.  LHA  7  construction  began  in 
July  2013  and  delivery  is  expected  in  December 
2018.  The  Navy  intends  to  use  limited  competition 
to  award  a  contract  for  LHA  8  in  fiscal  year  2016. 


Source:  Navy  photo  by  Mass  Communication  Specialist  1st  Class  Michael  McNabb. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Huntington  Ingalls 

As  of 

Latest 

Percent 

Industries 

01/2006 

07/2015 

change 

Program  office:  Washington,  DC 

Research  and  development  cost 

$237.0 

$446.6 

88.4% 

Funding  needed  to  complete: 

Procurement  cost 

$3,175.0 

$9,521.6 

199.9% 

R&D:  $38.3  million 

Total  program  cost 

$3,412.1 

$9,970.6 

192.2% 

Procurement:  $3,221.7  million 

Program  unit  cost 

$3,412,072 

$3,323,531 

-2.6% 

Total  funding:  $3,261 .2  million 

Total  quantities 

1 

3 

200.0% 

Procurement  quantity:  1 

Acquisition  cycle  time  (months) 

146 

182 

24.7% 

LHA  6  began  construction  in  December  2008  with 
mature  technologies  but  an  incomplete  design. 
The  ship  delivered  in  April  2014  after  a  20  month 
delay  and  has  begun  post-delivery  activities, 
including  the  post  shakedown  availability  which 
was  delayed  until  May  2015.  The  program 
extended  the  duration  of  this  activity  through 
February  2016  to  test  modifications  made  to  the 
flight  deck  of  the  ship  so  that  it  can  accommodate 
the  F-35.  LHA  7-which  largely  shares  the  LHA  6 
design-began  construction  in  July  2013. 
Changes  to  LHA  8  are  more  significant  and 
include  the  addition  of  a  well  deck.  The  program 
is  working  with  two  shipyards  and  intends  to 
competitively  award  a  contract  to  one  of  them  in 
fiscal  year  2016. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  LHA  6 


LHA  6  Program 

Technology,  Design,  and  Production  Maturity 

When  the  program  awarded  the  LHA  6  construction 
contract  in  June  2007,  it  considered  all  LHA  critical 
technologies,  plus  six  additional  non-critical 
subsystems  necessary  to  achieve  capabilities,  as 
mature.  The  Joint  Precision  Approach  and  Landing 
System,  a  subsystem,  was  restructured  to  focus  on 
aircraft  auto-land  capabilities,  and  it  is  uncertain 
when  the  system  will  be  ready  for  installation.  In  the 
interim,  the  program  will  use  backup  aviation  control 
systems  to  meet  requirements. 

LHA  6,  the  lead  ship,  was  delivered  to  the  Navy  in 
April  2014,  20  months  later  than  the  contracted 
delivery  date.  The  program  plans  to  integrate  17 
additional  design  changes  to  accommodate  the  F- 
35.  These  changes  are  estimated  to  cost  about  $57 
million. 

Construction  of  LHA  7-which  modifies  the  LHA  6 
design-began  in  July  2013,  and  over  30  percent  of 
the  ship  has  been  built  as  of  July  201 5.  The 
program  anticipates  the  ship's  delivery  will  be 
delayed  by  about  six  months  to  December  2018  due 
to  the  design  changes  required  for  operations  with 
the  F-35.  Although  the  program  claims  the 
shipbuilder  has  improved  its  performance  by 
implementing  lessons  learned  and  re-hiring  staff 
from  the  LHA  6  construction,  the  contractor  is 
currently  not  achieving  the  necessary  labor 
efficiency  rates  and  is  reporting  increasing  costs. 
The  Navy  has  provided  incentives  specific  to  the 
shipbuilder  to  promote  better  quality. 

Design  changes  on  LHA  8  will  be  more  significant 
as  the  Navy  is  incorporating  a  well  deck  that 
accommodates  two  landing  craft,  and  work  is 
underway  to  design  the  ship  to  reduce  the  ship's 
acquisition  and  lifecycle  costs.  The  Navy  is  working 
with  the  two  shipyards  that  it  determined  capable  of 
building  the  ship  without  major  recapitalization  to 
assist  with  this  effort. 

Other  Program  Issues 

LHA  6's  delivery  was  20  months  later  than  initially 
planned,  which  delayed  the  start  of  operational 
testing  10  months  to  April  2015.  In  order  to  create 
cost  savings  by  combining  operational  testing  with  a 
pre-deployment  fleet  exercise,  the  end  of 
operational  testing  was  extended  by  9  months  to 


June  2017.  In  order  to  make  modifications  to 
accommodate  the  F-35s,  the  Navy  recently 
extended  a  planned  maintenance  period  used  to 
complete  work  and  correct  defects.  The  Navy 
currently  plans  to  declare  the  ship  operational  in 
September  2016.  This  would  mean  the  Navy  plans 
to  declare  the  ship  operational  nearly  a  year  before 
operational  testing  is  complete. 

In  January  2015,  the  Director,  Operational  Testing 
and  Evaluation  expressed  concern  that  the  LHA  6's 
defense  system  will  not  be  able  to  defend  against  all 
anti-ship  cruise  missile  threats.  The  program  initially 
disagreed,  but  believes  it  has  taken  some  actions  to 
address  these  concerns. 

The  Navy  issued  a  combined  solicitation  to  award 
separate  contracts  for  the  detail,  design  and 
construction  of  the  LHA  8,  sixTAO(X)  oiler  ships, 
and  contract  design  support  for  the  LX(R)  program. 
Currently  the  Navy  plans  to  conduct  a  limited 
competition  between  two  shipbuilders  and  award 
the  LHA  8  to  one  competitor,  the  TAO(X)  to  a 
second  competitor,  and  portions  of  the  LX(R) 
contracts  to  each  competitor.  Navy  officials  believe 
that  this  strategy  will  both  support  the  industrial 
base  to  maintain  competition  for  future  projects,  and 
lower  prices. 

Program  Office  Comments 

LHA  6  was  successfully  delivered  with  no  fit  and 
finish  concerns  and  no  starred  trial  cards, 
outstanding  for  a  first  of  class  large  deck 
amphibious  ship,  and  is  designed  and  built  to  meet 
its  validated  performance  characteristics.  Recently, 
LHA  6  completed  a  two-month  operational 
deployment  with  Marine  Corps,  expeditionary  strike 
group  staff,  and  aircraft.  During  the  post  shakedown 
availability,  LHA  6  is  receiving  modifications  to  fully 
integrate  ship  capability  in  support  of  F-35B 
operations  in  preparation  for  operational 
deployment.  Declaration  of  LHA  6's  initial 
operational  capability  is  planned  for  September 
201 6  per  the  LHA  capability  development 
document. Construction  of  LHA  7  is  not  experiencing 
labor  resource  issues  as  seen  in  the  past  and  the 
shipbuilder  continues  to  drive  a  positive  schedule 
variance  for  the  program.  According  to  the  program, 
labor  efficiency  concerns  are  being  addressed  by 
the  shipbuilder  and  contract  incentives  have  worked 
to  improve  construction  quality. 
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Lead  Component:  Navy 


Common  Name:  LCS 


Littoral  Combat  Ship  (LCS) 

The  Navy's  Littoral  Combat  Ship  (LCS)  is  designed 
to  perform  mine  countermeasures,  antisubmarine 
warfare,  and  surface  warfare  missions.  It  consists  of 
the  ship  itself,  or  seaframe,  and  the  mission 
package  it  deploys.  The  Navy  is  buying  two  designs 
of  LCS-the  Freedom  variant,  a  steel  monohull  (LCS 
1  and  odd  numbered  ships)  and  the  Independence 
variant,  an  aluminum  trimaran  hull  (LCS  2  and  even 
numbered  ships)-and  has  awarded  contracts  for  26 
seaframes.  We  assessed  both  seaframe  designs. 


Source:  Lockheed  Martin  (left);  General  Dynamics  (right). 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Austal  USA, 

Lockheed  Martin 

Program  office:  Washington  Navy  Yard, 

Research  and  development  cost 

As  of 
05/2004 

$951.61 

Latest 

08/2015 

$3,464.5 

Percent 

change 

264.1% 

DC 

Procurement  cost 

$505.97 

$17,253.4 

3,310.0% 

Funding  needed  to  complete: 

Total  program  cost 

$1,457.58 

$20,953.5 

1,337.6% 

R&D:  $323.9  million 

Program  unit  cost 

$364,396 

$654,797 

79.7% 

Procurement:  $6,505.7  million 

Total  quantities 

4 

32 

700.0% 

Total  funding:  $6,966.8  million 

Acquisition  cycle  time  (months) 

41 

119 

190.2% 

Procurement  quantity:  9 

Cost  data  are  for  the  seaframe  program  only.  Research  and  development  funding  includes  detail 
design  and  construction  of  two  ships. 

The  LCS  seaframe  program  has  demonstrated 
the  maturity  of  16  of  its  18  critical  technologies 
and  has  accepted  delivery  of  six  seaframes.  The 
Navy  recently  downgraded  the  maturity 
assessments  of  two  critical  technologies, 
identified  critical  deficiencies  during  acceptance 
testing,  and  continues  to  incorporate  changes  into 
follow-on  ship  designs.  In  March  2015,  contract 
modifications  changed  the  delivery  dates  of  LCS 
5  through  24  by  up  to  a  year  or  more,  and  added 
priced  options  for  the  procurement  of  LCS  25  and 
26  and  special  incentives.  The  Navy  plans  to 
procure  a  future  small  surface  combatant- 
referred  to  as  a  frigate-based  on  a  modified  LCS 
design  starting  in  fiscal  year  2019. 


Page  105 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 


Demonstrate  all  critical  technologies  in  an  operational 
environment 


o 


Complete  preliminary  design  review 


Product  design  is  stable 


•  Complete  three-dimensional  product  model 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


9  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 

Not  applicable 
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Common  Name:  LCS 


LCS  Program 

Technology  Maturity 

Sixteen  of  the  18  critical  technologies,  the  total 
number  of  technologies  for  both  designs,  are 
mature  and  have  been  demonstrated  in  a  realistic 
environment.  The  Navy  downgraded  the  maturity  of 
two  Independence  variant  technologies-the 
aluminum  structure  and  the  launch,  handling  and 
recovery  system  planned  for  use  on  this  variant 
after  LCS  4.  The  Navy  reported  that  the  results  of 
survivability  and  seaframe  operational  testing  will 
validate  the  maturity  of  the  aluminum  structure.  The 
Navy  changed  the  vendor  for  the  launch,  handling, 
and  recovery  system  on  LCS  6,  according  to  the 
program  office,  and  certification  testing  is 
incomplete.  At  LCS  6's  acceptance  trials,  the  Navy 
reported  this  system  resulted  in  one  of  the  8  critical 
deficiencies  identified.  The  Chief  of  Naval 
Operations  approved  delivery  of  the  ship  in  August 
2015  based  on  a  plan  to  correct  these  deficiencies. 
The  Navy  also  accepted  LCS  5,  a  Freedom  variant, 
following  trials  in  September  2015.  Both  LCS  5  and 
6  will  undergo  full  ship  shock  trials  in  summer  2016 
and  a  survivability  trial  for  the  Independence  variant 
is  scheduled  for  January  2016.  The  Navy  formally 
declared  the  Independence  variant  as  capable  of 
initial  operations  in  December  2015. 

Design  and  Production  Maturity 

To  date,  the  Navy  has  accepted  delivery  of  six 
seaframes:  LCS  7  through  LCS  20  are  in  various 
stages  of  construction;  and  LCS  21  through  26  are 
now  under  contract.  In  March  2015,  the  Navy 
modified  the  block  buy  contracts  for  LCS  5  through 

24  to  add  priced  options  for  the  procurement  of  LCS 

25  and  26,  approved  delays  in  LCS  delivery 
schedules  by  up  to  a  year  or  more,  and  added  new 
incentives  totaling  up  to  $45  million  to  each  yard  for 
launch  and  delivery.  The  Navy  continues  to 
incorporate  changes  into  follow-on  ships,  including 
updated  radars  starting  on  LCS  17  and  an  over-the- 
horizon  surface-to-surface  missile.  The  program 
has  also  undertaken  efforts  to  reduce  weight 
including  removal  of  the  fin  stabilizer  system  from 
the  Freedom  variant  (LCS  9  and  following)  and 
reducing  fuel  on  the  Independence  variant  (LCS  4 
and  following). 


Other  Program  Issues 

In  February  2014,  the  office  of  the  Secretary  of 
Defense  directed  the  Navy  to  contract  for  no  more 
than  32  LCS,  citing  concerns  about  the  ship's 
survivability  and  lethality.  In  December  2014,  the 
same  office  accepted  the  Navy's  recommendation 
to  procure  a  modified  LCS-which  the  Navy  is  calling 
a  frigate-forthe  final  20  ships  of  its  52  small  surface 
combatants  requirement.  The  frigate  is  expected  to 
have  some  improved  lethality  and  survivability 
capabilities,  including  an  over-the-horizon  missile 
and  improved  sensors,  though  the  improvements 
will  not  offer  as  robust  a  capability  as  other  options 
considered  by  the  Navy.  The  Navy  is  concurrently 
developing  requirements,  design,  final  acquisition 
strategy,  and  a  service  cost  position.  The  feasibility 
of  upgrades  will  depend  on  continuing  weight 
reduction  initiatives  undertaken  by  the  LCS  program 
and  shipbuilders.  Delays  in  the  delivery  of  LCS  may 
impact  the  feasibility  of  the  Navy's  plan  to  start 
frigate  production  at  the  LCS  shipyards  in  fiscal  year 
2019.  In  December  2015,  the  Secretary  of  Defense 
directed  the  Navy  to  decrease  the  quantity  of 
seaframes  to  40.  The  program  office  is  in  the 
process  of  responding  to  this  direction. 

Program  Office  Comments 

In  addition  to  providing  technical  comments,  the 
program  office  noted  that,  as  of  December  2015, 
both  LCS  variants  have  achieved  initial  operational 
capability.  According  to  Navy  officials,  the  design  is 
stable,  meets  all  approved  requirements,  and  is  in 
full  serial  production  at  both  shipyards.  With  the 
delivery  of  LCS  5  and  6,  each  shipyard  will  deliver 
an  LCS,  on  average,  every  six  months  for  the 
remainder  of  the  block  buy.  LCS  5  and  6  delivered 
with  the  fewest  trial  cards,  or  issues  that  require 
correction,  for  each  variant  to  date.  The  Navy 
continues  to  validate  modeling  and  simulation 
through  testing.  Fiscal  year  2016  planned  test 
events  of  note  include:  multi-compartment  surrogate 
events;  full  ship  shock  trials;  and  launch,  handling, 
and  recovery  system  certification.  The  Navy  has 
assessed  cost,  schedule,  and  technical  feasibility  of 
forward-fit  frigate  lethality/survivability 
enhancements  on  the  Flight  0+  ships  and 
developed  plans  for  incorporation  as  early  as  fiscal 
year  2016,  given  the  appropriate  funding. 
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Lead  Component:  Navy 


Common  Name:  LCS  Packages 


Littoral  Combat  Ship  -  Mission  Modules  (LCS  Packages) 

The  Littoral  Combat  Ship  (LCS)  will  provide  mine 
countermeasures  (MCM),  surface  warfare  (SUW), 
and  antisubmarine  warfare  (ASW)  capability  using 
mission  packages.  Packages  include  weapons  and 
sensors  launched  and  recovered  from  LCS 
seaframes.  The  Navy  plans  to  deliver  capability 
incrementally  and  has  set  interim  requirements  that 
are  below  the  baseline  requirements  for  some 
increments.  We  assessed  mission  packages 
progress  against  the  threshold  requirements  that 
define  the  baseline  capabilities  currently  expected 
for  each  package. 

Source:  U.S.  Navy. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 
Systems  Corporation 

Program  office:  Washington  Navy  Yard, 

Research  and  development  cost 

As  of 
08/2007 

N/A 

Latest 

09/2015 

$2,520.4 

Percent 

change 

N/A 

DC 

Procurement  cost 

$3,496.8 

$4,377.3 

25.2% 

Funding  needed  to  complete: 

Total  program  cost 

N/A 

$6,930.0 

N/A 

R&D:  $657.3  million 

Program  unit  cost 

N/A 

$108,282 

N/A 

Procurement:  $3,763.4  million 

Total  quantities 

64 

64 

0.0% 

Total  funding:  $4,452.9  million 

Acquisition  cycle  time  (months) 

N/A 

N/A 

N/A 

Procurement  quantity:  49 

The  current  estimate  does  not  include  $3.6  billion  of  procurement  funding  for  life  cycle 
and  modernization  of  mission  systems. 

replacement 

The  Navy  has  accepted  delivery  of  13  mission 
packages  prior  to  demonstrating  that  they  meet 
threshold  capability  requirements  (threshold 
requirements)  on  both  seaframes.  The  MCM 
package  recently  completed  a  series  of 
development  tests  on  the  first  increment  of 
capability,  but  reliability  issues  have  led  the  Navy 
to  suspend  further  testing.  The  current  increment 
of  the  SUW  package  met  its  interim  requirements 
on  one  seaframe  and  is  in  testing  on  the  other. 
The  Navy  is  modifying  an  Army  missile  to  provide 
capabilities  to  meet  threshold  requirements  for 
SUW.  The  ASW  package  has  been  delayed  by  a 
year  as  funding  was  moved  from  ASW  to  MCM. 
The  Navy  is  developing,  testing,  and  integrating 
the  sub-systems  needed  to  meet  threshold 
requirements  for  each  mission  package  on  both 
seaframe  variants  and  is  not  expected  to  field  a 
set  of  mission  packages  that  meet  these 
requirements  until  2019. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 

•  Demonstrate  all  critical  technologies  in  an  operational 
environment 

•  Complete  preliminary  design  review  £ 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  LCS  Packages 


LCS  Packages  Program 

Mine  Countermeasures  (MCM) 

The  Navy  has  accepted  six  MCM  packages  without 
demonstrating  that  they  meet  interim  or  threshold 
requirements.  The  package  has  four  increments: 
the  first  is  designed  to  remove  sailors  from  the 
minefield  and  improve  mine  detection,  classification, 
and  neutralization  over  legacy  vessels.  Operational 
testing  for  the  first  increment  was  scheduled  to 
begin  in  fiscal  2015.  This  testing  has  been 
suspended  following  a  series  of  performance  and 
reliability  shortfalls  during  developmental  tests.  The 
Navy  stated  that,  when  the  package  was  available, 
it  significantly  exceeded  performance  requirements 
during  tests.  The  Department  of  Operational  Test 
and  Evaluation  stated  that  the  Navy  did  not  take  into 
account  that  the  systems  were  unavailable  for  85  of 
132  days  of  testing.  Test  officials  determined  that 
the  current  MCM  system  would  not  be  found 
operationally  effective  and  critical  MCM  systems 
and  the  Independence-variant  seaframe  are  not 
reliable.  Test  officials  support  the  Navy's  September 
2015  decision  to  suspend  further  testing  and 
evaluate  alternatives  to  key  systems  and  assess 
technical  and  programmatic  risks.  The  findings  of 
this  evaluation  have  not  yet  been  finalized. 

Surface  Warfare  (SUW) 

The  Navy  has  accepted  seven  SUW  packages  and 
plans  to  accept  one  more  in  fiscal  2017.  Each 
increment  one  package  currently  consists  of  two  30 
millimeter  guns,  an  armed  helicopter,  and  two  rigid 
hull  inflatable  boats.  In  August  2014,  the  Navy  found 
that  the  package  met  interim  performance 
requirements  on  the  Freedom  variant  and  is 
currently  testing  the  package  on  the  Independence 
variant.  To  meet  threshold  requirements  for  SUW  a 
surface-to-surface  missile  is  required.  The  Navy 
plans  to  use  the  Army's  Longbow  HELLFIRE  missile 
for  this  capability,  as  it  canceled  two  previous 
efforts.  According  to  program  officials,  initial 
demonstrations  with  Longbow  HELLFIRE  have 
been  successful  and  operational  testing  is  planned 
for  fiscal  year  2017. 

Antisubmarine  Warfare  (ASW) 

According  to  the  Navy,  the  systems  that  comprise 
the  ASW  mission  package  are  mature,  as  they  have 
been  fielded  by  United  States  and  foreign  navies.  In 
September  2014,  the  Navy  completed  development 
testing  aboard  the  Freedom  variant,  but  the  mission 


package  is  currently  5  tons  over  its  weight 
parameters.  Navy  program  officials  stated  that  they 
recently  awarded  contracts  to  reduce  package 
weight  by  at  least  15  percent.  The  Navy  is  now 
planning  to  meet  the  threshold  requirement  for  ASW 
in  2017,  a  one  year  delay  from  last  year's  estimate, 
as  the  Navy  redirected  funding  for  ASW  to  make  up 
for  funding  shortfalls  in  the  MCM  and  SUW 
packages. 

Other  Program  Issues 

The  Navy  continues  to  procure  LCS  seaframes, 
even  though  the  sub-systems  necessary  to  meet 
threshold  mission  package  requirements  have  not 
yet  been  fully  developed  and  integrated  with  both 
seaframe  designs.  The  Navy  will  not  achieve  the 
capability  to  meet  threshold  requirements  for  all 
three  of  the  mission  packages  until  2019,  by  which 
time  it  plans  to  take  delivery  of  22  ships.  The  Navy 
plans  to  begin  procurement  of  a  modified  LCS  in 
2019. 

Program  Office  Comments 

The  Navy  stated  that  it  is  purchasing  the  quantity  of 
mission  systems  and  packages  needed  for  system 
integration,  crew  training,  developmental  testing, 
operational  testing,  and  LCS  deployments.  The 
packages  have  all  been  demonstrated  in  a  relevant 
environment  prior  to  integration.  The  Navy  is 
purchasing  the  systems  in  accordance  with  DOD 
regulations.  The  Navy  is  following  its  plan  to 
incrementally  deliver  operationally  effective  mission 
package  capability  to  the  fleet  rather  than  waiting 
years  to  acquire  all  mission  systems  needed  to 
meet  minimum  requirements.  For  example,  initial 
SUW  capability  has  been  fielded  as  planned.  Full 
SUW  and  ASW  capability  will  be  fielded  in  fiscal 
2017.  The  program  office  provided  technical 
comments,  which  were  incorporated  as  appropriate. 

GAO  Response 

The  systems  that  comprise  the  Navy's  mission 
packages  have  yet  to  work  together  to  achieve 
stated  minimum  requirements.  The  failures  of  the 
MCM  package  during  testing  this  year  and  the 
subsequent  indefinite  delay  of  MCM  initial  capability 
are  emblematic  of  the  Navy's  challenges.  In  the 
absence  of  a  defined  increment-based  approach  to 
sequentially  gain  knowledge  and  meet 
requirements,  the  Navy's  acquisition  approach  is 
not  in  accordance  with  best  practices. 
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MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C  Triton) 


The  Navy's  MQ-4C  Triton  is  intended  to  provide 
persistent  maritime  intelligence,  surveillance,  and 
reconnaissance  (ISR)  data  collection  and 
dissemination  capability.  Triton  is  planned  to  be  an 
unmanned  aircraft  system  operated  from  five  land- 
based  sites  worldwide  as  part  of  a  family  of  maritime 
patrol  and  reconnaissance  systems.  Based  on  the 
Air  Force's  RQ-4B  Global  Hawk  air  vehicle,  Triton  is 
part  of  the  Navy's  plan  to  recapitalize  its  airborne 
ISR  assets  by  the  end  of  the  decade. 


Source:  Courtesy  Northrop  Grumman/Robert  Brown. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

As  of 

Latest 

Percent 

Systems  Corporation 

02/2009 

12/2014 

change 

Program  office:  Patuxent  River,  MD 

Research  and  development  cost 

$3,370.5 

$3,950.7 

17.2% 

Funding  needed  to  complete: 

Procurement  cost 

$10,002.5 

$8,563.9 

-14.4% 

R&D:  $228.4  million 

Total  program  cost 

$13,783.4 

$12,814.6 

-7.0% 

Procurement:  $8,497.1  million 

Program  unit  cost 

$196,906 

$183,066 

-7.0% 

Total  funding:  $8,866.7  million 

Total  quantities 

70 

70 

0.0% 

Procurement  quantity:  66 

Acquisition  cycle  time  (months) 

92 

120 

30.4% 

The  Triton  is  projected  to  enter  production  in  May 
2016  with  its  one  critical  technology  mature,  its 
design  stable,  and  critical  manufacturing 
processes  demonstrated  on  a  pilot  production 
line.  The  maturity  of  some  manufacturing 
processes,  however,  is  below  the  level 
recommended  by  best  practices.  There  have 
been  quality  and  schedule  delays  with  the 
manufacturing  of  Triton's  wings  for  test  aircraft, 
and  the  desired  manufacturing  process  maturity 
level  for  the  wings  will  not  be  achieved  until  after 
production  start.  Flight  testing  is  currently  behind 
schedule  and  fatigue  testing  has  been  delayed, 
resulting  in  concurrency  between  development 
and  production  where  discovering  deficiencies 
may  result  in  costly  rework  and  retrofits.  The  size 
and  complexity  of  the  aircraft's  software  continues 
to  increase  with  less  reuse  of  existing  code  than 
originally  planned. 


Demonstrate  critical  processes  are  in  control 
Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


o 

o 


9  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 

Not  applicable 
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MQ-4C  Triton  Program 

Technology  and  Design  Maturity 

The  program's  only  critical  technology  is  now  fully 
mature.  According  to  the  Navy,  Triton's  design  was 
considered  stable  at  the  critical  design  review  in 
2011,  but  demonstration  of  the  design  maturity 
through  testing  with  a  system-level  integrated 
prototype  did  not  occur  until  2012.  There  was 
immature  technology  at  program  start  and  no 
prototype  testing  at  the  design  review,  so  the 
program  accepted  knowledge  deficits.  In  addition, 
the  number  of  design  drawings  have  increased  by 
70  percent  since  2011  because  drawings  releasable 
at  that  time  did  not  include  drawings  for  components 
other  than  the  air  vehicle.  According  to  the  program, 
the  design  is  now  stable  and  demonstrated. 

Production  Maturity 

The  program  held  a  production  readiness  review  in 
June  2015  and  concluded  that  the  system  was 
ready  to  proceed  to  production  based  on  the 
demonstration  of  its  critical  manufacturing 
processes  on  a  pilot  production  line  and  other 
activities.  The  review  highlighted  key  areas  of 
remaining  work,  including  assessing  the 
producibility  of  Triton's  wings.  Triton  wing  quality 
problems  have  delayed  wing  delivery  for  some  test 
aircraft.  The  program  does  not  expect  to  bring  its 
wing  manufacturing  processes  under  control,  as 
recommended  by  best  practices,  until  about  six 
months  after  production  start.  In  addition,  the 
program  will  not  test  a  fully  configured,  production 
representative  prototype  to  demonstrate  system 
functionality  and  reliability,  also  recommended  by 
best  practices,  until  October  2016,  5  months  after 
production  start. 

Other  Program  Issues 

The  program  previously  planned  to  complete 
approximately  13,000  flight  test  points  prior  to 
December  2015.  As  of  September  2015,  fewer  than 
5,000  have  been  completed.  Overall,  more  than 
22,000  test  points  are  planned,  which  means  the 
Navy  will  be  executing  the  majority  of  its  test  points 
after  production  start.  The  Navy  also  delayed  the 
start  of  fatigue  testing  to  determine  the  airframe's 
long-term  durability  until  two  years  after  production 
start.  The  delivery  of  18  production  aircraft  is 
planned  before  this  testing  is  complete.  Our 
previous  work  has  shown  that  concurrency  between 
development  testing  and  production  increases  the 


risk  of  discovering  deficiencies  that  could  require 
costly  design  changes  to  systems  already 
produced.  The  Navy  acknowledges  that  the  Triton 
may  face  this  risk. 

The  size  and  complexity  of  Triton's  software 
continues  to  increase.  The  estimated  software  lines 
of  code  needed  has  increased  from  6.6  million  to 
8.9  million  since  the  start  of  software  development, 
with  the  estimated  amount  of  new  software 
increasing  from  16  to  26  percent.  Generally,  the 
development  of  new  code  takes  more  effort  than  the 
reuse  or  modification  of  existing  code. 

Finally,  the  Navy  recently  changed  Triton's 
acquisition  strategy  in  June  2015  by  combining 
three  separate  increments  into  one  acquisition 
effort.  As  we  reported  in  December  2014,  this  is  a 
riskier  approach  that  deviates  from  best  practices. 
Our  previous  work  has  found  that  evolutionary, 
incremental  approaches  facilitate  more  success  in 
meeting  program  goals.  According  to  acquisition 
guidance  in  place  at  the  time  of  the  Navy's  original 
business  case,  as  well  as  statute,  an  incremental 
approach  would  have  required  the  Navy  to  develop 
and  present  a  business  case  for  each  future 
increment.  We  recommended  in  our  previous  work 
that  the  Navy  update  the  existing  business  case; 
however,  the  program  has  not  yet  completed  this 
task. 

Program  Office  Comments 

In  commenting  on  a  draft  of  the  assessment,  the 
program  office  stated  that  the  MQ-4C  Triton 
program  continues  to  demonstrate  success  during 
its  system  development  and  demonstration  phase 
as  evidenced  by  successful  entry  into  operational 
assessment  on  November  17,  2015.  Production 
readiness  reviews  have  been  conducted  in 
preparation  for  the  low-rate  initial  production 
decision  review  in  the  third  quarter  of  fiscal  year 
2016.  While  the  program  has  experienced  delays  in 
wing  delivery  for  the  two  system  demonstration  test 
articles,  production  planning  is  on  schedule  for 
aircraft  delivery  in  support  of  initial  operational  test 
and  evaluation.  Upon  approval  of  the  acquisition 
strategy  on  June  15,  2015,  the  program  fully 
integrated  planning  for  phased  capability  upgrades. 
The  Triton  program  benefits  from  strong  support 
within  the  Department  of  the  Navy. 


Page  110 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


Lead  Component:  Navy 


Common  Name:  Fire  Scout 


MQ-8  (Fire  Scout) 

The  Navy's  MQ-8  unmanned  aerial  vehicle  is 
intended  to  provide  real-time  imagery  and  data  in 
support  of  intelligence,  surveillance,  and 
reconnaissance  missions.  The  MQ-8  system  is 
comprised  of  one  or  more  air  vehicles  with  sensors, 
a  control  station,  and  ship  equipment  to  aid  in 
launch  and  recovery.  The  air  vehicle  launches  and 
lands  vertically,  and  operates  from  ships  and  land. 

The  MQ-8  is  intended  for  use  in  various  operations, 
including  surface,  anti-submarine,  and  mine 
warfare.  We  assessed  the  latest  variant,  the  MQ- 
8C. 


Source:  Northrop  Grumman  Systems  Corporation. 
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Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 
Program  office:  Patuxent  River,  MD 
Funding  needed  to  complete: 

Research  and  development  cost 

As  of 
12/2006 

$641.6 

Latest 

12/2014 

$1,233.0 

Percent 

change 

92.2% 

R&D:  $129.8  million 

Procurement  cost 

$1,805.3 

$1,684.9 

-6.7% 

Procurement:  $731.6  million 

Total  program  cost 

$2,806.1 

$2,917.9 

4.0% 

Total  funding:  $861.5  million 

Program  unit  cost 

$15,854 

$41,685 

162.9% 

Procurement  quantity:  16 

Total  quantities 

177 

70 

-60.5% 

Acquisition  cycle  time  (months) 

104 

169 

62.5% 

Cost  and  quantity  data  are  for  development  and  procurement  of  both  the  MQ-8B  and  MQ-8C  variants. 

The  MQ-8  program,  which  consists  of  a  B  and  C 
variant,  was  restructured  in  2015  as  it  reported  a 
Nunn-McCurdy  unit  cost  breach  of  the  critical 
threshold.  According  to  program  officials,  the 
requirement  for  the  system  to  operate  from  the 
Littoral  Combat  Ship  was  updated  to  include  the 
MQ-8C  and  the  MQ-8B,  resulting  in  a  stoppage  of 
MQ-8B  production,  decreased  quantities,  and 
increased  unit  cost.  The  MQ-8C  is  a  larger 
airframe  but  shares  previously  developed 
technology  from  the  MQ-8B.  As  a  result  of  the 
Nunn-McCurdy  breach,  the  program's  schedule 
has  been  revised.  According  to  program  officials, 
the  program's  recertification  of  production  is 
scheduled  for  May  2016,  with  the  start  of 
operational  testing  in  September  2017  and  initial 
operational  capability  for  MQ-8C  in  April  2018. 


Attainment  of  Product  Knowledge 


As  of  January  2016 
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•  Demonstrate  all  critical  technologies  in  aan  operational 
environment 


Complete  preliminary  design  review 


Product  design  is  stable 


Release  at  least  90  percent  of  design  drawings 
Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


Demonstrate  critical  processes  are  in  control 
Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


9  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 

Not  applicable 
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Fire  Scout  Program 

Technology  and  Design  Maturity 

According  to  the  program  office,  MQ-8C  has  90 
percent  commonality  with  the  previously  developed 
MQ-8B,  the  primary  difference  being  structural 
modifications  to  accommodate  the  MQ-8C's  larger 
airframe  and  fuel  system.  The  MQ-8C  relies  on 
mature  technologies  common  to  the  MQ-8B  and 
has  completed  all  of  its  planned  engineering  design 
drawings  as  of  August  2014.  The  MQ-8C  was 
initiated  under  the  Navy's  rapid  deployment 
capability  procurement  process,  which  enabled  the 
program  to  bypass  many  standard  acquisition 
practices.  According  to  program  officials,  this 
process  traded  concurrency  risk  for  a  speedier 
acquisition  cycle  time.  First  flight  for  the  MQ-8C 
occurred  in  October  2013.  The  program  will 
continue  with  developmental  testing  and  transition 
to  operational  testing  in  the  third  quarter  of  fiscal 
year  2017.  Despite  the  separate  iteration  of 
development,  the  MQ-8C  did  not  have  a  separate 
system  development  start  decision,  which, 
according  to  program  officials,  was  not  required. 

Production  Maturity 

The  program  awarded  a  contract  for  development 
and  production  of  the  MQ-8C  in  April  2012  after 
production  of  the  MQ-8B  variant  stopped.  The 
program  has  subsequently  acquired  19  MQ-8C 
aircraft  using  the  Navy's  rapid  deployment  capability 
procurement  process.  According  to  program 
officials,  these  19  units  will  be  considered  low-rate 
initial  production  units  for  the  planned  May  2016 
production  recertification  decision.  There  will  be 
additional  low-rate  initial  production  units  procured 
after  this  May  2016  decision,  but  according  to 
program  officials  this  quantity  has  not  been 
determined.  We  could  not  assess  whether  critical 
manufacturing  processes  were  in  control,  as  the 
program  does  not  collect  data  on  statistical  process 
controls  or  assess  process  capabilities  using 
manufacturing  readiness  levels.  The  program  office 
collects  metrics  on  the  status  of  production  from  the 
prime  contractor,  such  as  discovery  of 
manufacturing  defects.  Program  officials  noted  that 
the  MQ-8C  air  vehicle  is  a  commercial  airframe 
procured  by  the  government  and  provided  directly  to 
the  prime  contractor  as  government  furnished 
equipment  for  conversion  to  a  MQ-8C.  The  prime 
contractor  is  responsible  for  integration  of  the 
avionics  and  working  with  the  aircraft  manufacturer 


to  modify  the  commercial  airframe  with  increased 
capacity  fuel  tanks,  new  electrical  systems,  and 
provisions  for  the  unmanned  avionics. 

Other  Program  Issues 

Current  issues  with  the  Littoral  Combat  Ship  have 
made  ship  availability  for  testing  with  the  MQ-8 
difficult.  According  to  program  officials,  continued 
difficulties  with  scheduling  may  affect  planned  initial 
operational  test  and  evaluation  dates.  Further, 
continued  software  integration  between  the  MQ-8C 
and  the  Littoral  Combat  Ship  also  depends  on  ship 
availability.  After  a  Nunn-McCurdy  unit  cost  breach 
of  the  critical  threshold,  in  June  2014  the  program 
was  certified  as  essential  to  national  security  and 
allowed  to  proceed.  The  restructured  program  plans 
to  seek  approval  of  the  new  acquisition  program 
baseline  and  recertification  of  the  production 
decision  in  May  2016. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  officials  stated  that  the  MQ-8  program  is 
executing  with  no  significant  issues.  A  land-based 
operational  assessment  of  the  MQ-8C  was 
completed  by  the  Navy's  Operational  Test  and 
Evaluation  Force  in  November  2015.  The  event 
successfully  completed  all  72  test  points  and 
objectives,  executing  82.4  flight  hours  in  17  days  at 
the  Point  Mugu  Test  Range.  The  MQ-8B  continues 
to  support  the  Littoral  Combat  Ship  with  on-going 
deployments. 
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Next  Generation  Jammer  Increment  1 

The  Navy's  Next  Generation  Jammer  (NGJ)  is  being  - 

developed  as  an  external  jamming  pod  system  fitted 
on  EA-18G  Growler  aircraft.  It  is  expected  to 
replace  the  ALQ-99  jamming  pod  system  and 
provide  enhanced  airborne  electronic  attack 
capabilities  to  disrupt  and  degrade  enemy  air 
defense  and  ground  communication  systems.  The 
Navy  plans  to  field  capabilities  in  three  increments 
for  different  radio  frequency  ranges,  beginning  with 
Increment  1  (mid-band)  in  2021,  with  Increments  2 
and  3  (low-  and  high-band)  to  follow.  We  assessed 
Increment  1. 

Source:  U.S.  Navy. 
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Program  Essentials 

Prime  contractor:  Raytheon 
Program  office:  Patuxent  River,  MD 
Funding  needed  to  complete: 

R&D:  $2,409.6  million 
Procurement:  $4,069.6  million 
Total  funding:  $6,483.3  million 
Procurement  quantity:  124 


Program  Performance  (fiscal  year  2016  dollars  in  millions) 


As  of 

Latest 

02/2015 

Percent 

change 

Research  and  development  cost 

N/A 

$3,325.9 

N/A 

Procurement  cost 

N/A 

$4,069.6 

N/A 

Total  program  cost 

N/A 

$7,399.7 

N/A 

Program  unit  cost 

N/A 

$57,810 

N/A 

Total  quantities 

N/A 

128 

N/A 

Acquisition  cycle  time  (months) 

N/A 

95 

N/A 

NGJ  plans  to  enter  system  development  in 
February  201 6,  with  its  seven  critical  technologies 
all  nearing  full  maturity.  Consistent  with  best 
practices  and  statutory  requirements,  the  program 
completed  a  preliminary  design  review  in  October 
2015  prior  to  its  planned  development  start. 
However,  risks  remain  in  several  areas,  in 
particular  meeting  weight  and  power 
requirements  and  integrating  the  jamming  pods 
with  the  aircraft.  The  NGJ  program  has  taken 
multiple  steps  to  reduce  design  and  integration 
risks  prior  to  development  start,  but  it  does  not 
plan  to  implement  one  key  best  practice- 
demonstrating  a  system-level  prototype  prior  to  its 
critical  design  review.  In  September  2015,  the 
Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  approved  the  NGJ 
Increment  1  to  participate  in  a  pilot  program  that 
allows  it  to  streamline  and  tailor  some  acquisition 
processes. 
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Common  Name:  NGJ  Increment  1 


NGJ  Increment  1  Program 

Technology  Maturity 

The  NGJ  program  plans  to  enter  system 
development  in  February  2016,  with  its  critical 
technologies  approaching  full  maturity.  In  November 
2015,  the  program  conducted  an  assessment  of  its 
seven  critical  technologies,  which  include  two 
separate  arrays-each  with  different  transmit/receive 
modules,  circulators,  and  apertures-as  well  as  a 
power  generation  system.  All  technologies  were 
assessed  as  nearing  maturity  based  on  subsystem 
prototyping  testing  conducted  prior  to  system 
development  start. 

Design  Maturity 

According  to  program  officials,  the  Navy  has 
identified  and  taken  steps  to  mitigate  several  key 
design  and  integration  risks.  The  program  considers 
achieving  the  necessary  power  within  weight 
constraints  to  be  the  greatest  risk.  Additional  risks 
include  integration  of  the  NGJ  with  the  EA-18G  and 
the  potential  for  electromagnetic  radiation  to  affect 
the  reliability  of  missiles  employed  on  the  EA-18G. 
The  program  has  taken  several  steps  to  mitigate 
these  risks.  According  to  officials,  the  program 
instituted  efforts  to  reduce  the  weight  of  multiple 
subsystems,  included  incentive  fees  related  to 
weight  and  power  in  the  development  contract,  and 
received  approval  for  a  higher  pod  weight  by 
making  trades  with  the  EA-18G.  In  addition,  the 
program  has  established  working  groups  to  address 
NGJ  and  EA-18G  software  and  hardware 
interoperability. 

The  NGJ  program  has  taken  multiple  steps  to 
reduce  design  risk  prior  to  development  start,  but  it 
does  not  plan  to  implement  one  key  best  practice 
intended  to  demonstrate  the  maturity  of  the 
system's  design  at  its  critical  design  review. 
Consistent  with  best  practices  and  statutory 
requirements,  the  program  completed  a  preliminary 
design  review  in  October  2015  prior  to  its  planned 
development  start.  Holding  preliminary  design 
review  before  the  start  of  system  development  helps 
ensure  that  requirements  are  defined  and  feasible 
and  the  proposed  design  can  meet  those 
requirements  within  cost,  schedule,  and  other 
system  constraints.  The  program  does  not  plan  to 
test  an  early  system  prototype  of  the  jamming  pod 
before  its  March  2017  critical  design  review.  Best 
practices  call  for  such  testing  to  take  place  before 


the  critical  design  review  to  help  determine  whether 
the  design  is  stable  and  will  work  as  intended.  The 
first  test  of  a  fully  functional  jamming  pod  is  planned 
for  March  2019.  Program  officials  believe  that  the 
significant  prototyping  and  associated  testing 
conducted  to  date  addresses  the  principal  program 
risks,  including  how  to  meet  key  performance 
requirements  within  design  constraints,  such  as  the 
size  and  weight  of  the  pod. 

Other  Program  Issues 

In  September  2015,  the  Undersecretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics  approved 
NGJ  as  the  first  program  in  the  Skunk  Works  pilot, 
which  aims  to  eliminate  non-value  added  processes 
in  order  to  deliver  capabilities  on  time  and  within 
budget.  The  Navy  and  DOD  are  streamlining 
documentation  and  review  processes  and  have 
delegated  documentation  approval  authority  to 
lower  levels  where  appropriate.  For  example,  the 
program  streamlined  the  process  for  conducting  its 
technology  readiness  assessment  by  reviewing  key 
test  results  with  DOD  and  Navy  representatives  and 
documenting  those  results  in  a  memo  instead  of  a 
formal  Technology  Readiness  Assessment  report. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
NGJ  program  said  it  has  designed  and  resourced 
extensive  prototype  testing,  focused  on  technical 
risks,  which  has  led  to  early  discovery  of  issues, 
minimized  risks  for  the  development  phase,  and  put 
the  program  on  solid  footing  for  production  and 
deployment.  Prototype  testing  was  conducted  at  the 
component-level  and  subsystem-level.  This  testing 
informed  the  preliminary  design.  The  program 
stated  that  it  also  leveraged  model  performance 
predictions,  which  were  informed  by  and  updated  as 
the  system  progressed  through  testing.  According 
to  the  program,  primary  prototype  testing  is 
complete  and  test  results  indicated  effective 
isotropic  radiated  power  and  prime  power 
requirements  were  met.  The  program  noted 
changes  from  prototype  designs  to  preliminary 
design  are  being  minimized  to  further  reduce 
system  development  risk.  Technical  comments  were 
provided  by  the  program  office,  which  were 
incorporated  where  deemed  appropriate. 
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Lead  Component:  Navy 


Common  Name:  OASuW  Inc  1 


Offensive  Anti-Surface  Warfare  Increment  1  (OASuW  Inc  1) 


The  Offensive  Anti-Surface  Warfare  Increment  1 
(OASuW  Inc  1)  is  a  Navy-led  program  to  develop  an 
air-launched,  long-range,  anti-surface  warfare 
missile  to  address  an  urgent  operational  need.  The 
program  is  using  an  accelerated  acquisition 
approach  and  is  leveraging  previous  technology 
demonstration  efforts.  It  plans  to  field  an  early 
operational  capability  on  Air  Force  B-1  bombers  in 
2018  and  Navy  F/A-18  aircraft  in  2019.  DOD  is 
developing  requirements  for  Increment  2  to  address 
the  threat  in  2024  and  beyond. 


Source:  U.S.  Navy. 
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Program  Essentials 

Prime  contractor:  Lockheed  Martin 
Program  office:  Patuxent  River  NAS, 
MD 

Funding  needed  to  complete: 

R&D:  $761.9  million 
Procurement:  $311.9  million 
Total  funding:  $1,073.8  million 
Procurement  quantity:  110 


Program  Performance  (fiscal  year  2016  dollars  in  millions) 


As  of 

Latest 

Percent 

06/2015 

change 

Research  and  development  cost 

N/A 

$1,111.7 

N/A 

Procurement  cost 

N/A 

$311.9 

N/A 

Total  program  cost 

N/A 

$1,423.6 

N/A 

Program  unit  cost 

N/A 

$11,481 

N/A 

Total  quantities 

N/A 

124 

N/A 

Acquisition  cycle  time  (months) 

N/A 

31 

N/A 

The  latest  cost  and  quantity  data  are  for  fiscal  years  2013  through  2019  for  the  acquisition  of 
Increment  1  capability.  The  timeline  dates  depict  when  equivalent  decision  reviews  will  occur. 

In  order  to  meet  urgent  operational  needs,  the 
OASuW  Inc  1  program  plans  to  enter  system 
development  in  February  2016  with  less 
knowledge  than  recommended  by  best  practices. 
Only  one  of  its  six  critical  technologies  is  nearing 
full  maturity  the  other  technologies  are  expected 
to  be  near  maturity  by  the  planned  June  2016 
critical  design  review.  The  program  plans  to 
release  more  than  90  percent  of  the  drawings  by 
critical  design  review  but  this  estimate  may  not 
include  all  the  subsystems  being  developed.  The 
accelerated  acquisition  approach  includes 
concurrency  between  developmental  testing  and 
production  which  increases  the  risk  of  late  design 
changes  and  costly  retrofits  after  production  has 
started.  The  program  is  attempting  to  mitigate  this 
concurrency  risk  by  leveraging  airframe 
commonality  with  the  JASSM-ER  missile  and 
through  early,  targeted  sub-system  testing. 


Attainment  of  Product  Knowledge 


Projected  as  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 

environment  ^ 

•  Demonstrate  all  critical  technologies  in  an  operational 
environment 

•  Complete  preliminary  design  review  0 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


0  Knowledge  attained  ■*■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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OASuW  Inc  1  Program 

Technology  Maturity 

OASuW  Inc  1  is  scheduled  to  enter  system 
development  in  February  2016  with  none  of  its  six 
critical  technologies  mature  and  one  approaching 
full  maturity  which  is  not  consistent  with  best 
practices.  Only  algorithms  for  electro-optical 
sensing  have  been  demonstrated  in  a  relevant 
environment.  The  program's  single  hardware-based 
critical  technology-a  radio  frequency  sensor-and 
four  other  software-based  ones-autorouter,  low 
altitude  control/sea  state  estimator,  multi-target 
tracker,  and  simultaneous  time  of  arrival 
algorithms-are  immature.  Program  officials 
estimate  that  all  critical  technologies  will  be 
approaching  maturity  prior  to  critical  design  review 
in  June  2016,  but  delays  may  occur  if  the  low 
altitude  control  algorithm  requires  further 
development. 

Design  and  Production  Maturity 

Although  it  is  proceeding  with  immature 
technologies,  the  program  completed  a  preliminary 
design  review  in  October  2014  prior  to  entering 
system  development.  The  program  estimates  that 
97  percent  of  design  drawings  will  be  releasable  by 
critical  design  review.  Although  a  recent 
independent  systems  engineering  review  found  that 
the  program's  design  appears  to  be  progressing 
with  a  well  understood  critical  path  and  actively 
mitigated  risks,  it  called  for  better  linkage  between 
requirements  and  design  and  noted  that  design 
drawing  estimates  may  not  include  all  mission 
systems  under  development. 

OASuW  Inc  1  builds  upon  the  knowledge  gained  by 
the  Defense  Advanced  Research  Projects  Agency’s 
(DARPA)  Long  Range  Anti-Ship  Missile  (LRASM) 
program-which  included  three  test  flights-and 
leverages  the  airframe  and  production  facilities  of 
the  Joint  Air-to-Surface  Standoff  Missile-Extended 
Range  (JASSM-ER)  program.  However,  the 
JASSM-ER  fuze-a  planned  component  of  the 
OASuW  missile  design-is  not  qualified  for  use  on 
Navy  weapons.  The  program  is  seeking  approval  to 
accept  the  associated  risk.  If  not  granted, 
development  of  an  alternate  fuze  may  cause  delays. 
An  OASuW-specific  production  maturity 
assessment  will  be  conducted  in  support  of  a 
production  readiness  review  scheduled  for 
November  2016. 


Other  Program  Issues 

Maintaining  the  program's  schedule  is  the  primary 
concern  for  OASuW  Inc  1  as  it  intends  to  address 
an  urgent  operational  need.  The  program's  current 
accelerated  acquisition  approach  consists  of 
decision  points  that  align  with  key  systems 
engineering  reviews,  test  events,  contract  actions, 
and  fielding  decisions.  The  approach  requires 
concurrency  between  initial  production  and 
developmental  testing,  which  entails  certain  risks. 
Our  past  work  has  shown  that  beginning  production 
before  demonstrating  that  a  design  is  mature  and 
will  work  as  intended  increases  the  risk  of 
discovering  deficiencies  during  production  that 
could  require  design  changes,  costly  modifications, 
and  retrofits.  The  program  has  accepted  this  risk 
and  mitigated  it,  in  part,  by  limiting  the  program  to 
the  110  missiles  needed  to  provide  a  capability  until 
the  second  increment  is  fielded.  According  to 
program  officials,  requirements  stability  for 
Increment  1  must  be  maintained  to  keep  capabilities 
at  a  level  that  satisfies  warfighter  needs  and 
mitigates  potential  schedule  risks  from  additional 
development  and  design  activities.  Program  officials 
state  that  they  have  been  successful  in  removing 
capabilities  found  to  be  extraneous  to  meeting 
operational  requirements. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  Navy 
officials  noted  that  OASuW  Inc  1  exists  to  fill  an 
urgent  need  and  emerging  capability  gap  and  to 
pace  the  growing  maritime  threat.  They  added  that 
timing  to  field  OASuW  capability  is  paramount.  As  a 
result,  the  acquisition  strategy  was  planned  with 
concurrent  system  qualification  and  test,  which 
does  introduce  risk.  To  reduce  risk  the  program 
leveraged  the  mature  JASSM-ER  platform,  with 
which  88  percent  is  common,  and  demonstrated 
technology  maturity  during  three  successful  DARPA 
flight  tests.  According  to  the  program,  LRASM's 
hardware  design  is  also  mature  with  technical 
margins  that  minimize  the  risk  of  later  hardware 
changes.  LRASM-unique  components  also 
leverage  legacy  systems  to  reduce  technical  and 
schedule  risk.  Finally,  OASuW  Inc  1  has  progressed 
from  program  initiation  through  critical  design 
review  in  18  months  and  is  on  track  to  field  on 
schedule.  The  Navy  also  provided  technical 
comments,  which  were  incorporated  as  appropriate. 
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Common  Name:  SSC 


Ship  to  Shore  Connector  Amphibious  Craft  (SSC) 


The  Navy's  SSC  is  an  air-cushioned  landing  craft 
intended  to  transport  personnel,  weapon  systems, 
equipment,  and  cargo  from  amphibious  vessels  to 
shore.  SSC  is  the  replacement  for  the  Landing 
Craft,  Air  Cushion,  which  is  approaching  the  end  of 
its  service  life.  The  SSC  is  designed  to  deploy  in 
Navy  well  deck  amphibious  ships,  such  as  the  LPD 
17  class,  and  for  use  in  assault  and  nonassault 
operations.  The  program  entered  system 
development  in  July  2012  and  held  its  low-rate 
production  decision  in  May  2015. 


Source:  U.S.  Navy. 
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Program  Essentials 

Prime  contractor:  Textron  Inc. 
Program  office:  Washington,  DC 
Funding  needed  to  complete: 
R&D:  $17.9  million 
Procurement:  $3,391.7  million 
Total  funding:  $3,429.5  million 
Procurement  quantity:  69 
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Program  Performance  (fiscal  year  2016  dollars  in  millions) 

As  of  Latest  Percent 

07/2012  08/2015  change 

Research  and  development  cost 

$597.2 

$525.6 

-12.0% 

Procurement  cost 

$3,624.4 

$3,541.1 

-2.3% 

Total  program  cost 

$4,241.6 

$4,086.5 

-3.7% 

Program  unit  cost 

$58,104 

$55,980 

-3.7% 

Total  quantities 

73 

73 

0.0% 

Acquisition  cycle  time  (months) 

135 

135 

0.0% 

The  SSC  program  entered  production  in  May 
2015,  with  its  critical  technology  mature  and 
design  stable,  but  before  demonstrating  that 
critical  processes  were  in  control.  The  program 
recently  completed  design  changes  to  the  drive 
train  gearbox,  which  is  among  the  items  the 
program  has  identified  as  potential  risk  areas. 
Other  risks  remain,  including  operational 
challenges  posed  by  the  steep  angle  of  the  craft's 
loading  ramp  and  low  productivity  in  software 
development.  According  to  program  officials,  SSC 
did  not  meet  two  required  production  exit  criteria 
and  plans  to  have  eight  craft  under  contract-with 
four  starting  fabrication-prior  to  delivery  of  the 
test  craft,  posing  risks  if  issues  are  identified 
during  testing.  A  planned  4  year  block  buy 
contract  starting  in  fiscal  year  2017  will  result,  if 
approved,  in  authorizing  33  craft  before  the  test 
craft  is  delivered. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 


•  Demonstrate  all  critical  technologies  in  an  operational 
environment 


•  Complete  preliminary  design  review 

• 

1  Product  design  is  stable 

•  Release  at  90  percent  of  design  drawings 

• 

•  Test  a  system-level  integrated  prototype 

o 

1  Manufacturing  processes  are  mature 

•  Demonstrate  critical  processes  are  in  control 

o 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

•  Test  a  production-representative  prototype 

o 

^  Knowledge  attained  ■  ■  ■  ■ 

Information  not  available 

O  Knowledge  not  attained 

Not  applicable 
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SSC  Program 

Technology  and  Design  Maturity 

The  program's  single  critical  technology,  the  fire 
suppression  system,  is  mature  and  the  design 
stable.  The  program  recently  completed  design 
changes  due  to  concerns  with  accelerated  wear  of 
the  drive  train  gearbox.  The  gearbox  was  not  ready 
for  factory  acceptance  testing  prior  to  production, 
but  the  program  believes  that  recent  design 
changes  will  address  the  wear  issues. 

In  May  2015,  the  Navy's  commander  of  operational 
test  and  evaluation  reported  that  the  steep  angle  of 
SSC's  loading  ramp  may  create  operational  hazards 
for  certain  vehicle  types.  The  program  will  not  have 
an  opportunity  to  fully  assess  this  and  other  issues 
until  the  completion  of  developmental  testing  in 
December  2017,  6  months  earlier  than  previously 
planned  due  to  a  recent  decision  to  use  two  craft  for 
testing-100  and  101-instead  of  one. 

Development  of  command,  control,  communication, 
computer  and  navigation  system  software  has  been 
less  productive  than  planned,  leading  Navy  cost 
estimators  to  find  that  software  development  costs 
would  likely  significantly  exceed  current  estimates, 
and  that  the  fixed-price  software  development 
contract  will  only  partially  mitigate  the  budgetary 
impact  of  this  cost  growth. 

Production  Maturity 

The  program  was  approved  for  low-rate  production 
in  May  2015,  3  months  later  than  planned. 
According  to  program  officials,  at  the  time  of  the 
decision  the  SSC  had  not  met  two  of  the  1 3  exit 
criteria  required  to  enter  low-rate  production-both 
related  to  the  gearbox-but  was  planning  to  meet 
those  remaining  criteria  by  January  2016.  Program 
officials  stated  that  the  program  has  experienced 
some  production  delays,  but  that  it  is  still  within  2  to 
3  months  of  its  schedule.  Fabrication  of  the  test  and 
training  craft  began  in  November  2014  and  was  16 
percent  complete  as  of  July  201 5.  Delivery  is 
projected  for  May  2017  with  delivery  of  the  second 
craft  in  August  2017.  The  delivery  of  the  test  craft  is 
critical  because  it  is  the  first  opportunity  to 
demonstrate  whether  capabilities  meet 
requirements  and  that  no  redesign  work  is  needed 
before  committing  to  production  of  additional  craft. 

The  program  currently  plans  to  have  eight  craft 
under  contract-with  four  starting  fabrication-prior  to 
delivery  of  the  test  craft.  Our  previous  work  has 


shown  that  such  concurrency  can  increase  the  risk 
of  discovering  deficiencies  in  testing  that  could 
require  costly  design  changes  and  modifications  to 
systems  already  produced.  As  a  result  of  the 
program's  concurrent  testing  and  production 
schedule,  it  will  not  demonstrate  that  the  craft 
function  reliably  in  an  operational  environment  until 
fiscal  year  2018.  Any  deficiencies  that  are 
discovered  would  require  correction  on  the  eight 
craft  already  in  production. 

Other  Program  Issues 

The  SSC  program  office  is  considering  a  single 
block  buy  contract-  which  is  typically  similar  to  a 
multiyear  contract,  but  without  the  same  statutory 
limitations-for  fiscal  years  2017  through  2020,  fora 
total  of  33  craft.  Program  officials  believe  this 
approach,  which,  if  this  authority  is  enacted  by 
Congress,  would  provide  more  stability  for  the 
vendors  and  reduce  costs  compared  to  an  annual 
procurement.  However,  committing  to  a  contract  for 
several  years  of  production  before  delivering  the 
test  craft  introduces  additional  risk.  If  issues 
requiring  design  changes  are  identified  during 
developmental  testing-not  scheduled  for 
completion  until  the  end  of  2017-the  program  could 
face  costly  rework  of  craft  already  in  production. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment, 
program  officials  stated  that  the  SSC  schedule, 
while  containing  some  concurrency,  is  the  most 
affordable  approach,  leveraging  production 
efficiencies  while  addressing  rising  sustainment 
costs  for  the  current  fleet.  The  program  is  mitigating 
concurrency  by  use  of  incremental  decision  points 
before  craft  awards,  design  maturity,  and  rigorous 
testing.  Eliminating  concurrency  would  drive 
increased  sustainment  funding  and  create  a 
production  break,  risking  loss  of  workforce 
expertise,  increasing  manufacturing  overhead  costs 
and  reducing  opportunities  for  economic  quantity 
buys.  Regarding  the  gearbox,  this  last  production 
exit  criterion  will  be  complete  with  the  first  article 
test  that  starts  in  January  2016.  Regarding  software 
cost,  revised  projections  suggest  lower  costs. 
Forecasts  in  2014  presumed  coding  inefficiencies 
that  have  not  materialized  to  date.  Regarding  the 
ramp,  the  SSC  geometry  is  similar  to  existing  craft. 
An  initial  assessment  was  completed  in  August 
2015.  A  final  technical  solution  is  under 
development. 
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Lead  Component:  Navy 


Common  Name:  VH-92A 


VH-92A  Presidential  Helicopter  Replacement  Program 

The  Navy's  VH-92A  program  provides  new 
helicopters  for  executive  transport  of  the  President, 

Vice  President,  heads  of  state,  and  others.  A 
successor  to  the  VH-71  program-canceled  due  to 
cost  growth,  schedule  delays,  and  performance 
shortfalls-the  planned  fleet  of  23  VH-92A  aircraft 
will  replace  23  legacy  helicopters.  The  VH-92A  is 
expected  to  provide  improved  operational 
performance  and  communications  capabilities, 
while  offering  increased  passenger  payload.  Until 
VH-92As  are  available,  the  Navy  is  ensuring  legacy 
fleet  availability. 


Source:  Sikorsky  Aircraft  Corporation. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Sikorsky  Aircraft 

As  of 

Latest 

Percent 

Corporation 

04/2014 

07/2015 

change 

Program  office:  Patuxent  River,  MD 

Research  and  development  cost 

$2,684.8 

$2,689.3 

0.2% 

Funding  needed  to  complete: 

Procurement  cost 

$2,104.8 

$2,127.5 

1.1% 

R&D:  $2,014.7  million 

Total  program  cost 

$4,789.5 

$4,816.8 

0.6% 

Procurement:  $2,127.5  million 

Program  unit  cost 

$208,241 

$209,428 

0.6% 

Total  funding:  $4,142.3  million 

Total  quantities 

23 

23 

0.0% 

Procurement  quantity:  17 

Acquisition  cycle  time  (months) 

76 

75 

-1.3% 

The  VH-92A  is  using  an  existing  platform  and 
incorporating  mature  technologies.  While  no  new 
technologies  are  involved,  the  fully  configured 
mission  communications  system  has  yet  to  be 
tested  in  an  aircraft.  Initial  ground  and  flight 
testing  consisting  of  antenna  co-site  interference 
and  system  performance  has  been  conducted. 
The  preliminary  design  review  was  completed  in 
August  2015,  15  months  after  the  start  of  system 
development.  Prior  to  this,  the  Navy,  along  with 
the  users,  reduced  risk  through  the  requirements 
management  process.  The  contractor  also 
recommended  alternative  icing  test  methods.  As 
of  December  2015,  the  contractor  completed  60 
percent  of  its  design  drawings  and  plans  to 
release  over  90  percent  by  critical  design  review 
scheduled  for  July  2016.  The  contractor  is 
producing  two  engineering  development  models 
to  perform  contractor  testing  in  2017  and  Navy 
testing  in  2018. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


•  Release  at  least  90  percent  of  design  drawings 


•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

9  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  VH-92A 


VH-92A  Program 

Technology  and  Design  Maturity 

The  VH-92A  program  began  system  development  in 
April  2014  with  its  mission  communications  system 
(MCS),  a  new  system  with  technology  critical  to  the 
mission,  approaching  full  maturity.  The  program  is 
modifying  a  commercially  available  aircraft  for  the 
VH-92A,  using  upgraded  technologies  that  are 
already  on  the  legacy  fleet.  The  sole  exception  is 
the  government  designed  MCS.  According  to 
officials,  on-site  government  program  monitors 
identified  a  potential  information  assurance  risk  that 
could  delay  the  delivery  of  MCS-a  subcomponent 
lacks  the  certification  documents  needed  to  ensure 
information  is  safeguarded  to  standards.  Another 
potential  risk  for  the  program  is  aircraft  weight 
growth.  For  example,  the  MCS  and  a  companion 
system  exceeded  their  allowed  weight  limit.  The 
contractor  implemented  a  weight  improvement 
program  and  reduced  the  weight  to  address  this 
problem.  The  system-level  preliminary  design 
review  was  held  in  August  2015,  15  months  after 
system  development  start,  as  the  program  received 
a  waiver  to  the  statutory  requirement  for  holding  it 
before  this  milestone.  The  preliminary  design  review 
resulted  in  12  actions  that  require  an  approved 
response.  Eleven  of  the  12  have  been  closed  and 
the  last  one-stabilization  of  a  communication 
system  component  to  prevent  vibrations  is  planned 
for  closure  in  early  2016-has  yet  to  be  approved.  In 
September  2015,  two  baseline  S-92A  aircraft  were 
inducted  into  the  contractor's  facility  to  commence 
modifications  and  become  the  two  engineering 
development  aircraft,  4  months  earlier  than 
planned.  One  had  previously  been  used  to  test 
antenna  placement  and  to  conduct  MCS  test 
activities. 

As  of  December  2015,  the  contractor  released  60 
percent  of  the  engineering  design  drawings.  The 
contractor  plans  to  release  90  percent  by  the  critical 
design  review,  scheduled  for  July  2016.  These 
drawings  represent  the  new  or  modified  equipment 
on  the  aircraft. 

Production  Maturity 

The  program  does  not  plan  to  collect  statistical 
process  control  data  for  critical  manufacturing 
processes,  but  will  instead  use  manufacturing 
readiness  levels.  The  program  plans  to  reach 
DOD’s  recommended  level  of  manufacturing 


process  maturity  for  the  start  of  production,  but  not  a 
level  that  indicates  processes  are  in  control  as 
recommended  by  best  practices. 

Other  Program  Issues 

The  Navy  determined  through  its  requirements 
management  process  that  it  no  longer  needed  the 
contractor  to  meet  a  certain  requirement.  Costs  to 
design  and  implement  a  capability  for  that 
requirement  were  included  in  the  negotiated  target 
contract  price,  as  this  capability  is  not  included  in 
the  baseline  S-92A  aircraft.  According  to  program 
officials,  the  capability  is  unnecessary  due  to  the 
helicopter's  operating  conditions.  The  Navy 
requested  Sikorsky  to  develop  and  submit  a  credit 
proposal  for  a  contract  price  adjustment  to  account 
for  the  change.  The  contractor's  original  plan  to 
conduct  flight  testing  to  meet  the  requirement  for 
flight  operations  in  icing  conditions  has  been  revised 
to  eliminate  the  flight  tests.  The  baseline  S-92A  is 
currently  certified  for  flight  operations  in  icing 
conditions,  and  the  contractor  has  determined  that 
the  VH-92A  configuration  can  be  certified  through 
modeling  and  analysis.  The  schedule  savings 
resulting  from  the  elimination  of  the  icing  tests  is 
being  applied  to  other  program  activities. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  report,  the  program 
office  noted  that  the  planned  VH-92A  fleet  of  23 
aircraft  includes  21  operational  aircraft  and  2  test 
aircraft,  the  engineering  development  model  aircraft 
being  produced  in  the  engineering  and 
manufacturing  development  phase  of  the  program. 
The  legacy  fleet  includes  two  test  aircraft  and 
recently  received  two  dedicated  training  aircraft, 
bringing  their  total  to  23  aircraft.  During  calendar 
year  2015,  the  program  completed  its  development 
milestones  as  planned  and  cost  estimates  are  within 
the  baseline  approved  at  the  system  development 
start  review.  According  to  program  officials,  the 
program  is  on  track  to  complete  its  critical  design 
review  later  this  year  as  planned.  The  program 
office  also  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Lead  Component:  Navy 


Amphibious  Ship  Replacement  (LX(R)) 


Common  Name:  LX(R) 


The  Navy's  LX(R)  program  plans  to  build  a  new 
class  of  ships  to  replace  existing  amphibious  ships. 
The  primary  function  of  these  ships  is  to  transport 
U.S.  Marines  and  their  equipment  to  distant 
operating  areas  and  enable  expeditionary 
operations  ashore.  The  LX(R)  will  include  a  larger 
hull  than  the  retiring  ships  and  can  also  be  used  for 
non-combat  operations  due  to  its  storage  space  and 
ability  to  transfer  people  and  supplies.  Starting  in 
fiscal  year  2020,  the  Navy  plans  to  procure  11  ships 
with  delivery  of  the  first  LX(R)  in  2026. 


Source:  Computer  Science  Corp  (CSC)  pursuant  to  LX(R)  Design  Support  Contract. 


Current  Status 

Based  on  an  analysis  of  alternatives,  the  Navy  decided  in  October  2014  to  base  the  LX(R)  design  on  the 
San  Antonio  (LPD  17)  class  amphibious  ships.  The  Navy  stated  that  by  modeling  the  LX(R)  on  an  existing 
ship  it  would  leverage  existing  design  maturity  and  reduce  the  cost  risks  associated  with  a  new  design.  Navy 
officials  stated  that  there  are  no  new  critical  technologies  planned.  Since  the  existing  LPD  17  design  is 
considered  unaffordable  for  the  LX(R)  program,  the  Navy  plans  to  remove  some  features  to  significantly 
reduce  LPD  17  ship  costs  while  still  ensuring  that  the  new  design  satisfies  all  LX(R)  requirements. 

The  Navy  is  using  a  limited  competition  approach  that  combines  LX(R)  design  efforts  with  acquisition 
activities  for  the  next  America  Class  Amphibious  Assault  Ship  (LHA  8)  and  Fleet  Replenishment  Oiler  (T- 
AO(X)),  to  maintain  stability  within  the  shipbuilding  industrial  base.  The  Navy  issued  a  combined  solicitation 
in  June  2015  that  was  limited  to  two  contractors-Huntington  Ingalls  Industries  and  General  Dynamics 
NASSCO-for  detail  design  and  construction  of  T-AO(X)  and  LHA  8  ships,  as  well  as  early  stage  design  work 
for  LX(R).  The  Navy  was  appropriated  $250  million  in  fiscal  year  201 6  for  advanced  procurement,  funds  that 
will  be  used  for  engineering,  and  planning  for  the  lead  LX(R)  ship,  and  plans  to  award  a  contract  for 
construction  in  2020.  The  acquisition  strategy  for  this  phase  of  the  program  is  not  finalized  and  the 
shipbuilders  who  will  compete  to  build  LX(R)  have  not  yet  been  determined.  The  program's  entrance  into 
system  development,  as  well  as  the  designation  of  the  milestone  decision  authority,  will  be  made  following 
the  approval  of  the  program's  requirements. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2016  dollars): 

Total  program:  $17,130  million 
Research  and  development:  $168  million 
Procurement:  $16,962  million 
Quantity:  11 

Next  Major  Program  Event:  Requirements  validation,  fiscal  2016 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  Navy  provided  technical 
comments  which,  were  incorporated  as  appropriate. 
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Lead  Component:  Navy 


Common  Name:  DDG  51  Flight  III 


DDG  51  Flight  III  Destroyer  (DDG  51  Flight  III) 

The  DDG  51  Flight  III  destroyer  will  be  a 
multimission  ship  designed  to  operate  against  air, 
surface,  and  subsurface  threats.  Compared  to 
existing  Flight  IIA  ships,  Flight  III  will  provide 
increased  ballistic  missile  and  area  air  defense 
capabilities.  Planned  configuration  changes  for 
Flight  III  include  replacing  the  current  SPY-1  D(V) 
radar  with  the  new  Air  and  Missile  Defense  Radar 
(AMDR).  The  Navy  plans  to  acquire  a  total  of  22 
Flight  III  ships  beginning  in  fiscal  year  2016. 


Source:  Bath  Iron  Works. 


Current  Status 

The  Navy  continues  Flight  III  design  activities,  including  the  award  of  the  detail  design  contracts  in  February 
2015  and  completion  of  a  preliminary  design  review  in  September  2015.  Flight  III  and  AMDR  development  is 
concurrent,  which  could  affect  ship  construction  if  delays  occur  in  AMDR  development  or  production.  In 
addition  to  AMDR,  Flight  III  changes  include,  among  other  things,  upgrades  to  the  ship's  electrical  plant. 
Power  conversion  modules  and  new  generators  similar  to  those  developed  for  the  DDG  1000  class  are  to 
supply  the  increased  power  needed  for  AMDR.  The  DDG  1000  program  had  issues  in  developmental  testing 
of  its  electrical  system,  and  additional  modifications  are  required  for  Flight  III.  The  Flight  III  design  also 
includes  configuration  changes  to  increase  weight  and  stability  margins,  which  determine  how  much  new 
equipment  can  be  incorporated  into  the  ship.  The  Navy  stated  that  it  believes  that  the  new  equipment's 
additional  weight  will  not  adversely  affect  Flight  III  performance,  but  the  ship's  internal  space  is  a  risk  that  it 
is  monitoring.  The  addition  of  AMDR  and  system  upgrades  will  account  for  the  majority  of  the  increased 
margins  and  may  limit  the  ability  to  introduce  future  upgrades. 

The  Navy  plans  to  modify,  using  an  engineering  change  proposal,  the  existing  Flight  IIA  multiyear 
procurement  contracts  in  order  to  construct  the  first  three  Flight  III  ships  rather  than  awarding  new  contracts. 
In  February  2015,  the  Navy  submitted  a  report  to  Congress  on  the  engineering  change  proposal,  reiterating 
that  the  low  to  moderate  risks  associated  with  AMDR  and  the  proposed  system  upgrades  justifies  its 
execution  within  the  next  year.  The  Navy  plans  to  award  construction  of  the  first  Flight  III  ship  in  the  third 
quarter  of  fiscal  year  2016  and  two  follow-on  ships  in  fiscal  year  2017. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2016  dollars): 

Total  program:  $43,381.5  million 
Research  and  development:  $4,038.3  million 
Procurement:  $39,343.2  million 
Quantity:  22  (procurement) 

Next  Major  Program  Event:  Award  of  first  Flight  III  engineering  change  proposal,  fiscal  year  2016. 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  Navy  noted  that  ship  design 
and  AMDR  engineering  and  manufacturing  development  remain  on  track. 
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Fleet  Replenishment  Oiler  (T-AO(X)) 


Common  Name:  T-AO(X) 


The  T-AO(X)  program  is  intended  to  replace  the 
Navy's  15  existing  T-AO  187  Class  Fleet  Oilers, 
which  are  nearing  the  end  of  their  service  lives.  Its 
primary  mission  is  to  provide  replenishment  of  bulk 
petroleum  products,  dry  stores/packaged  cargo,  fleet 
freight,  mail,  and  personnel  to  other  vessels  while 
underway.  The  Navy  plans  to  procure  a  total  of  17 
ships,  starting  with  the  first  T-AO(X)  in  fiscal  year 
201 6  and  the  remaining  ships  at  a  rate  of  one  per 
year  beginning  in  fiscal  year  2018. 


Source:  U.S.  Navy.  Notional  Concept  T-AO(X) 


Current  Status 

The  Navy  has  completed  cost  and  capabilities  trade  studies,  which  suggest  that  T-AO(X)  will  be  able  to  meet 
minimum  capability  requirements  within  projected  costs  utilizing  commercial  ship  designs  and  technologies. 
The  Navy  plans  to  employ  military  unique  systems  for  specific  functions,  such  as  underway  replenishment. 
According  to  an  October  2014  technology  readiness  assessment,  the  program's  three  critical  technologies 
are  fully  mature  based  on  the  results  of  land-based  and  at-sea  prototype  testing.  The  most  notable  of  these 
technologies,  Heavy-Electric  Standard  Tension  Replenishment  Alongside  Method,  allows  the  transfer  of 
cargo  at  double  the  standard  speed  or,  alternatively,  double  the  standard  load  weight.  These  technologies 
are  required  to  meet  the  more  robust  requirements  for  underway  replenishment  associated  with  the  new 
Ford-class  aircraft  carriers.  Due  to  its  plans  to  use  a  commercial  design,  the  program  was  granted  a  waiver 
for  the  requirement  to  conduct  competitive  prototyping  and  plans  to  request  a  waiver  for  preliminary  design 
review  prior  to  the  start  of  system  development. 

After  approval  in  June  2015,  the  Navy  released  its  request  for  proposals  to  two  companies  under  a  tailored 
T-AO(X)  acquisition  strategy.  The  Navy  is  using  a  limited  competition  acquisition  strategy  that  combines 
T-AO(X)  with  acquisition  activities  for  its  amphibious  ships  LHA  and  LX(R),  which  it  believes  will  maintain 
stability  within  the  shipbuilding  industrial  base.  The  Navy  anticipates  awarding  a  fixed-price  incentive 
contract  in  May  2016  for  lead  ship  detail  design  and  construction.  Navy  officials  indicated  that  the  remaining 
ships  will  be  acquired  at  a  rate  of  one  ship  per  year  under  two  additional  contracts,  possibly  using  firm  fixed- 
price  contracts.  The  program's  estimated  costs  and  buy  profile  beyond  fiscal  year  2020  are  under 
development. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2016  dollars): 

Total  program  (fiscal  years  2011-2020):  $2,381  million 
Research  and  development  (fiscal  years  2011-2014):  $56.1  million 
Procurement  (fiscal  years  2016-2020):  $2,324.9  million 
Quantity:  4 

Next  Major  Program  Event:  System  development  start,  May  2016 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 
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Common  Name:  OR 


OHIO-CLASS  REPLACEMENT  (OR) 


The  Navy's  Ohio-class  Replacement  (OR)  will 
replace  the  current  fleet  of  Ohio-class  ballistic 
missile  submarines  (SSBNs)  as  they  begin  to  retire 
in  2027.  The  Navy  began  technology  development  in 
January  2011  in  order  to  avoid  a  gap  in  sea  based 
strategic  deterrence  between  the  Ohio-class's 
retirement  and  the  production  of  a  replacement. 

Costs  for  the  OR  program  are  expected  to  account 
for  approximately  one-sixth  of  the  Navy's 
shipbuilding  budget  for  the  next  30  years,  even  as 
annual  shipbuilding  budgets  rise  above  the  historical 
average. 

Source:  ©  2012  General  Dynamics  Electric  Boat. 


Current  Status 

The  program  is  developing  and  conducting  tests  of  a  prototype  interface  for  the  OR's  ship  control  system 
and  other  new  technologies  while  focusing  on  production  readiness.  The  Navy  released  the  Integrated 
Process  and  Product  Development  request  for  proposals  (RFP)  in  January  2016,  and  anticipates  awarding 
the  contract  by  October  2016.  Prior  to  the  start  of  system  development  in  August  2016,  the  program  plans  to 
hold  a  preliminary  design  review  to  validate  the  design  is  stable.  Development  start  will  be  followed  by  the 
release  of  the  RFP  for  long  lead  time  materials  planned  for  September  2018.  According  to  program  officials, 
construction  has  begun  on  two  testing  facilities:  one  to  test  the  missile  fire  control  systems  using  production 
representative  systems  and  one  to  test  the  systems  that  eject  the  missile  from  the  launch  tube.  Both 
shipyards  are  using  an  improved  computer-based  design  tool  which  speeds  the  transition  from  design  to 
construction  and  helps  develop  an  efficient  build  strategy.  The  lead  shipyard  is  finishing  construction  of  a 
representative  hull  section  to  validate  the  fidelity  of  the  new  design  tool.  The  program  plans  to  complete  the 
three-dimensional  model  prior  to  starting  ship  construction,  an  important  step  in  reducing  the  risk  of  cost 
growth  as  identified  by  GAO's  best  practices  for  shipbuilding.  The  lead  shipyard  has  also  constructed  a  new 
manufacturing  building  and  is  installing  new  manufacturing  fixtures. 

According  to  program  officials,  the  Joint  Requirements  Oversight  Council  validated  the  OR's  requirements  in 
August  2015.  The  program  office  continues  to  investigate  cost  saving  options.  One  option,  which  would 
require  additional  congressional  approval,  includes  leveraging  Virginia  class  and  OR  acquisitions  using  a 
multiyear  procurement  to  facilitate  economic  order  quantity  purchases  across  both  submarine  programs. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2016  dollars): 

Total  program:  $97,021.2  million 
Research  and  development:  $11,954.5  million 
Procurement:  $85,066.7  million 
Quantity:  12 

Next  Major  Program  Event:  System  development  start,  August  2016 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  Navy  provided  technical 
comments,  which  were  incorporated  where  appropriate. 
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Lead  Component:  Navy 


Common  Name:  P-8A 


P-8A  Increment  3 


The  Navy's  P-8A  Increment  3  program  is  intended  to 
provide  enhanced  capabilities  to  the  P-8A  aircraft  in 
two  blocks.  The  first  block  is  to  include 
communications,  radar,  and  weapons  upgrades, 
which  will  be  incorporated  into  the  existing  P-8A 
architecture  as  a  series  of  discrete  engineering 
change  proposals.  The  second  block  will  establish 
new  open  systems  architecture  and  integrate 
improvements  to  the  combat  system's  ability  to 
process  and  display  classified  information  and  its 
search,  detection,  and  targeting  capabilities. 


Source:  U.S.  Navy. 


Current  Status 

In  October  201 3,  the  Undersecretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  approved  the  P- 
8A  Increment  3  program  to  bypass  technology  development  and  begin  the  acquisition  process  in  system 
development,  which  is  planned  to  begin  in  the  third  quarter  of  fiscal  year  201 7.  The  program  plans  to 
continue  development  of  the  P-8A  Increment  3  capabilities  as  engineering  change  proposals,  but  it  will 
conduct  an  overall  preliminary  design  review  in  2017.  As  the  program  bypassed  technology  development,  it 
has  not  yet  developed  an  independent  cost  estimate. 

The  P-8A  Increment  3  program  expects  to  deliver  improved  capabilities  to  the  P-8A  aircraft,  while 
introducing  competition  and  increasing  government  responsibilities  as  a  part  of  future  upgrades.  Program 
officials  stated  that  Increment  3  will  require  the  integration  of  new  hardware  and  software,  but  these 
capabilities  will  be  based  on  mature  technologies  and  no  critical  technologies  have  been  identified.  The 
Navy  awarded  sole-source  contracts  to  Boeing-the  P-8A  prime  contractor-to  integrate  Block  1  capabilities 
into  the  existing  aircraft.  For  Block  2,  the  Navy  is  developing  applications  based  on  open  systems 
architecture  as  an  extension  of  the  existing  mission  systems  architecture,  which  will  allow  it  to  openly 
compete  the  development  and  integration  of  future  combat  system  capabilities.  This  architecture  upgrade  is 
being  competitively  prototyped  and  executed  as  an  open  source,  collaborative  development.  The  Navy, 
which  plans  to  act  as  systems  integrator,  awarded  two  software  development  contracts  in  2014  to  Lockheed 
Martin  and  Raytheon.  Program  officials  said  they  will  choose  the  best  parts  of  each  design  during  system 
development  and  integrate  them  into  a  single  architecture.  The  hardware  will  be  funded  as  a  part  of  the 
operating  and  support  costs  of  the  P-8A. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2016  dollars): 

Total  program:  $1,003.6  million 
Research  and  development:  $1,003.6  million 
Procurement:  N/A 
Quantity:  109 

Next  Major  Program  Event:  Development  request  for  proposal  release  decision,  third  quarter  fiscal  year 
2016 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  program  officials  stated  that  the 
P-8A  Increment  3  program  continues  to  prepare  acquisition  documentation  and  the  requests  for  proposals 
for  future  contracts  that  will  support  the  integration  of  these  future  capabilities  into  the  P-8A  aircraft  in  a 
timely  and  cost  effective  manner. 
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Lead  Component:  Navy 


Common  Name:  UCLASS 


Unmanned  Carrier-Launched  Airborne  Surveillance  and  Strike 

(UCLASS)  System 


The  Navy's  UCLASS  system  is  expected  to  address 
a  gap  in  persistent  sea-based  intelligence, 
surveillance,  reconnaissance,  and  targeting  (ISR&T) 
and  provide  precision  strike  capabilities.  The  system 
is  made  up  of  three  key  segments:  an  air  segment,  a 
carrier  segment,  and  a  control  system  and 
connectivity  segment. 


Source:  Jeff  Hobrath,  Chief,  USN  (Ret)  President,  Naval  Tees,  LLC. 


Current  Status 

Release  of  the  request  for  proposals  for  the  design,  fabrication,  test,  and  delivery  of  the  air  system  continues 
to  be  delayed  pending  the  outcome  of  a  review  of  all  of  DOD's  ISR&T  programs.  According  to  program 
officials,  while  this  review  was  completed  in  July  2015,  no  decision  about  UCLASS  was  made  at  that  time, 
and  discussions  are  expected  to  continue  until  the  Office  of  the  Secretary  of  Defense's  (OSD)  fiscal  year 
2017  budget  submission. 

GAO  reported  in  May  2015  that  the  intended  mission  and  required  capabilities  of  UCLASS  were  under 
review,  as  there  is  ongoing  debate  within  OSD  about  whether  the  primary  role  of  the  UCLASS  system 
should  be  surveillance  with  limited  strike  capability  or  strike  with  limited  surveillance  capability.  This  debate 
has  delayed  the  program.  Award  of  the  air  system  development  contract  is  now  expected  in  fiscal  year  2017, 
a  delay  of  about  3  years.  The  Navy  now  expects  to  achieve  early  operational  capability-defined  as  a 
UCLASS  system  on  at  least  one  aircraft  carrier-in  fiscal  year  2022,  a  delay  of  about  2  years. 

GAO  also  reported  that  knowledge  the  Navy  has  obtained  in  developing  UCLASS,  including  from  reviewing 
the  preliminary  designs  of  four  contractors,  may  no  longer  be  completely  applicable  depending  on  what 
requirements  are  finally  chosen.  If  the  final  UCLASS  requirements  emphasize  a  strike  role  with  limited 
surveillance,  the  Navy  will  likely  need  to  revisit  the  areas  of  technology,  design,  and  funding  which  could 
further  delay  the  award  of  an  air  system  development  contract.  GAO  therefore  recommended  that  before 
committing  significant  resources,  the  Navy  should  ensure  that  it  has  an  executable  business  case  for 
UCLASS  development  that  matches  available  resources  to  the  required  capabilities.  The  Navy  generally 
agreed  with  the  recommendation. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2016  dollars): 

Total  program:  TBD 

Research  and  development  (fiscal  years  2012-2020):  $3,043  million 
Procurement:  TBD 
Quantity:  TBD 

Next  Major  Program  Event:  Release  request  for  proposals  for  design,  fabrication,  test,  and  delivery  of  air 
system. 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  Navy  provided  technical 
comments,  which  were  incorporated  as  appropriate. 
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Air  Force  Service  Summary 


We  performed  in-depth  assessments  on  11  of  the  26  Air  Force  major  defense  acquisition  programs  in  the 
current  portfolio  that  for  the  most  part  are  in  system  development  or  the  early  stages  of  production.  We  also 
assessed  seven  programs  identified  as  future  major  defense  acquisition  programs  that  are  expected  to  enter 
system  development  in  the  next  few  years.  The  Air  Force  currently  estimates  a  need  of  more  than  $103  billion 
in  funding  to  complete  the  acquisition  of  these  programs.  The  programs  in  system  development  or  production, 
where  we  determined  cost  and  schedule  change  from  first  full  estimates,  have  experienced  more  than  $39 
billion  in  cost  growth  and  average  schedule  delays  of  approximately  18  months.  More  of  the  cost  growth  for  Air 
Force  programs  occurred  in  the  past  5  years  than  prior  to  that  time,  primarily  due  to  cost  changes  on  the 
Evolved  Expendable  Launch  Vehicle  program.  Of  these  same  programs,  four  have  completed  all  the  activities 
associated  with  the  applicable  knowledge-based  best  practices  we  assess. 


Acquisition  Phase  and  Size  of  the  Currently  Estimated  Acquisition  Cost 

1 8  Programs  Assessed  for  the  1 8  Programs  Assessed 


.GPS  OCX 


Fiscal  year  2016  dollars  in  billions 


Programs  in 

technology 
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Programs 
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development 


Cost  and  Schedule  Growth  on  10  Programs 
in  the  Current  Portfolio 


Fiscal  year  2016  dollars  in  billions 
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Research  and  Procurement  Total 

development  costs  acquisition 

costs  costs 


□  Growth  from  first 
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estimate 


Summary  of  Knowledge  Attained  to  Date  for 
Programs  Beyond  System  Development  Start 
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KP  2 
in  future 

KP  3 
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KC-46A 

o 

• 

KP  3 
in  future 

SDB  II 

• 

• 

• 

Space  Fence  Inc  1 

O 

3DELRR 

o 

KP  2 
in  future 

KP  3 
in  future 

•  All  applicable  knowledge  practices  were 
completed 


One  or  more  applicable  knowledge  practices 
were  not  completed 

All  knowledge  practices  were  not  applicable 
Information  not  available  for  one  or  more 
knowledge  practice 


•  Cost  growth  of  more  than  15  percent  and/or 
schedule  delays  of  more  than  6  months 

•  Cost  growth  of  15  percent  or  less  and  schedule 
delays  of  6  months  or  less 

No  first  full  estimate  available 


Average 
acquisition 
cycle  time 

(in  months) 


91 


Note:  Bubble  size  is  based  on  each  program’s  currently  estimated 
future  funding  needed. 


Note:  In  addition  to  research  and  development  and  procurement 
costs,  total  acquisition  cost  includes  acquisition  related  operations 
and  maintenance  and  system-specific  military  construction  costs. 


Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 
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Air  Force  Program  Assessments 

Page 

2-page  assessments  number 

B-2  Defensive  Management  System  Modernization  (B-2  DMS-M)  129 

Combat  Rescue  Helicopter  (CRH)  131 

Enhanced  Polar  System  (EPS)  133 

Evolved  Expendable  Launch  Vehicle  (EELV)  135 

F-22  Increment  3.2B  Modernization  (F-22  Inc  3.2B  Mod)  137 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T)  139 

Global  Positioning  System  III  (GPS  III)  141 

KC-46  Tanker  Modernization  Program  (KC-46A)  143 

Military  GPS  User  Equipment  (MGUE)  Increment  1  145 

Next  Generation  Operational  Control  System  (GPS  OCX)  147 

Small  Diameter  Bomb  Increment  II  (SDB  II)  149 

Space  Fence  Ground-Based  Radar  System  Increment  1  (Space  Fence  Inc  1)  151 

Three-Dimensional  Expeditionary  Long-Range  Radar  (3DELRR)  153 

1-page  assessments 

Advanced  Pilot  Training  (APT)  155 

F-15  Eagle  Passive  Active  Warning  Survivability  System  (F-15  EPAWSS)  156 

Joint  Surveillance  Target  Attack  Radar  System  Recapitalization  (JSTARS  Recap)  157 

Presidential  Aircraft  Recapitalization  (PAR)  158 

Weather  Satellite  Follow-on  (WSF)  159 
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Lead  Component:  Air  Force 


Common  Name:  B-2  DMS-M 


B-2  Defensive  Management  System  Modernization  (B-2  DMS-M) 

The  Air  Force’s  B-2  DMS-M  program  is  expected  to 
upgrade  the  aircraft’s  1980s-era  analog  defensive 
management  system  to  a  digital  capability.  The 
program  intends  to  improve  the  frequency  coverage 
and  sensitivity  of  the  system,  update  pilot  displays, 
and  enhance  in-flight  replanning  capabilities  to 
avoid  unanticipated  air  defense  threats.  It  also 
expects  to  improve  the  reliability  and  maintainability 
of  the  DMS  system  and  the  B-2’s  readiness  rate. 


Source:  U.S.  Air  Force. 
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Low-rate  Full-rate  Initial 

decision  decision  capability 

(6/19)  (10/21)  (6/22) 


Program  Essentials 

Prime  contractor:  Northrop  Grumman 
Corp 

Program  office:  Wright-Patterson  AFB, 
OH 

Funding  needed  FY  2016  to  FY  2020 
R&D:  $941.2  million 
Procurement:  $292.4  million 
Total  funding:  $1,233.6  million 
Procurement  quantity:  20 


Program  Performance  (fiscal  year  2016  dollars  in  millions) 


As  of 

Latest 

10/2015 

Percent 

change 

Research  and  development  cost 

N/A 

$1,671.9 

N/A 

Procurement  cost 

N/A 

$272.4 

N/A 

Total  program  cost 

N/A 

$1,944.3 

N/A 

Program  unit  cost 

N/A 

N/A 

N/A 

Total  quantities 

N/A 

20 

N/A 

Acquisition  cycle  time  (months) 

N/A 

130 

N/A 

The  latest  cost  data  reflect  only  the  funding  planned  for  fiscal  years  2011  through  2020  and  not  a  cost 
to  complete  amount  as  the  acquisition  program  baseline  has  not  been  approved. 

The  B-2  DMS-M  program  is  expecting  to  enter 
system  development  in  January  2016  with  one 
critical  technology  fully  mature  and  three  critical 
technologies  nearing  full  maturity.  Consistent  with 
best  practices  and  statute,  the  program 
completed  a  preliminary  design  review  prior  to 
development  start.  The  Air  Force  invested  about 
40  percent  of  its  total  expected  DMS-M 
development  funds  during  technology 
development,  completing  activities  such  as 
software  coding  and  ordering  some  test 
hardware.  However,  risks  remain  in  several  areas 
including  system  integration.  The  Air  Force 
expects  to  achieve  initial  operating  capability  in 
2022.  Test  aircraft  availability,  test  range 
scheduling,  test  data  turnaround  times,  and 
contractor  staffing  may  present  challenges  to 
meeting  this  schedule. 
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Attainment  of  Product  Knowledge 


Projected  as  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 


Demonstrate  all  critical  technologies  in  an  operational 
environment 


o 


Complete  preliminary  design  review 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


Demonstrate  critical  processes  are  in  control 
Demonstrate  critical  processes  on  a  pilot  production  line 


•  Test  a  production-representative  prototype 


9  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 

Not  applicable 
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Common  Name:  B-2  DMS-M 


B-2  DMS-M  Program 

Technology  and  Design  Maturity 

The  DMS-M  program  expects  to  enter  system 
development  with  one  critical  technology  fully 
mature,  the  receiver-processor  for  bands  1-3,  based 
on  flight  tests  from  another  program  that  uses  a 
similar  system.  Three  other  critical  technologies  are 
nearing  full  maturity,  having  been  demonstrated  in  a 
relevant  environment.  When  the  program  entered 
the  technology  development  phase  in  2011 ,  the 
band  4  antenna,  band-4  receiver-processer,  and 
geo-location  algorithm  technologies  were  very 
immature  with  limited  demonstrations  completed  at 
that  time.  Since  then,  the  program  office  has  worked 
with  the  operational  users  to  refine  requirements 
and  better  match  them  with  the  maturity  of  the 
technologies.  As  a  result,  B-2  DMS-M  capabilities 
have  been  reduced,  but  the  user  has  determined 
that  it  is  an  acceptable  trade  given  the  additional 
cost  and  time  that  would  be  needed  to  meet  the 
more  demanding  requirements.  While  the  band  4 
receiver-processor  technology  has  been 
demonstrated  on  other  programs,  the  team  that 
independently  assessed  the  DMS-M  program  noted 
that  it  still  poses  a  risk  due  to  the  aircraft’s  intended 
operating  environment. 

Consistent  with  best  practices  and  statute,  the 
program  completed  a  preliminary  design  review 
prior  to  beginning  system  development.  The  Air 
Force  invested  almost  40  percent  of  its  total 
expected  DMS-M  development  funds  during  the 
technology  development  phase  to  reduce  risk.  Risk 
reduction  activities  completed  during  technology 
development  included  ordering  flight  test  hardware 
and  extensive  software  coding.  Competitive 
prototyping  was  conducted  for  the  band  4  antennas 
and  some  subsystems  completed  critical  design 
reviews.  The  DMS-M  contractor  estimates  that  80  to 
90  percent  of  the  software  development  is 
complete.  Technical  challenges  remain  as  the 
program  enters  system  development  including,  the 
installed  antenna  impact  on  the  aircraft’s  low 
observable  features,  calibration  of  the  DMS-M 
system  with  installed  antenna  data,  and  the 
completion  of  software  development.  The  program 
office  currently  projects  that  two  key  performance 
parameters  could  be  at  risk,  but  it  expects  to 
mitigate  these  risks  over  the  next  4  years. 


Other  Program  Issues 

Initial  operational  capability  has  been  delayed  from 
2019  to  2022  and  estimated  costs  have  increased 
by  $500  million  since  estimates  made  at  the  start  of 
the  technology  development  phase  in  201 1 . 
Program  officials  attribute  some  of  this  delay  to 
funding  reductions  that  occurred  from  fiscal  year 
2013  to  2015.  The  program  office  has  identified 
several  initiatives  that  it  hopes  will  lower  costs  by 
as  much  as  $189  million.  These  include  using 
different  test  ranges  to  improve  test  sortie  rates  and 
bounding  the  system  development  phase  to  5 
years  by  incentivizing  the  contractor  to  meet 
schedules.  There  are  challenges  that  could  impact 
the  program’s  ability  to  meet  schedules,  including 
the  availability  of  a  single  test  aircraft,  test  range 
availability  and  data  turnaround  times,  and 
contractor  staffing. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  comments,  which  were 
incorporated  where  deemed  appropriate. 
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Lead  Component:  Air  Force 


Common  Name:  CRH 


Combat  Rescue  Helicopter  (CRH) 

The  Air  Force's  Combat  Rescue  Helicopter  (CRH)  is 
intended  to  recover  personnel  from  hostile  or  denied 
territory  as  well  as  conduct  humanitarian,  civil 
search  and  rescue,  disaster  relief,  and  non- 
combatant  evacuation  missions.  The  CRH  program 
is  an  effort  to  replace  aging  HH-60G  Pave  Hawk 
helicopters.  The  first  effort  to  replace  the  HH-60G, 
the  Combat  Search  and  Rescue  Replacement 
Vehicle  (CSAR-X),  was  canceled  in  2009  due  to 
cost  concerns  stemming  from  technology 
development. 


Source:  ©2015  Sikorsky  Aircraft  Corporation.  Used  with  permission  for  support  of  CRH. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Sikorsky  Aircraft 

As  of 

Latest 

Percent 

Corporation 

06/2014 

06/2015 

change 

Program  office:  Wright-Patterson  AFB, 

Research  and  development  cost 

$2,017.9 

$2,018.5 

0.0% 

OH 

Procurement  cost 

$6,292.8 

$6,542.6 

4.0% 

Funding  needed  to  complete: 

Total  program  cost 

$8,335.1 

$8,588.0 

3.0% 

R&D:  $1,757.9  million 

Program  unit  cost 

$74,421 

$76,679 

3.0% 

Procurement:  $6,542.6  million 

Total  quantities 

112 

112 

0.0% 

Total  funding:  $8,326.7  million 

Acquisition  cycle  time  (months) 

75 

82 

9.3% 

Procurement  quantity:  108 

The  CRH  program  began  system  development  in 
June  2014  without  identifying  critical  technologies 
and  received  waivers  to  multiple  program 
certifications,  including  those  for  preliminary 
design  review  and  technology  risk  reduction. 

DOD  authorized  these  waivers  as  the  program 
plans  to  modify  an  existing  helicopter  with  mature 
subsystems  and  software.  A  technology 
readiness  assessment  is  now  underway  and  the 
program  plans  to  conduct  a  preliminary  design 
review  in  2016.  Program  officials  stated  that 
testing  to  demonstrate  the  design  will  not  occur 
prior  to  the  critical  design  review.  Although  the 
CRH  design  is  intended  to  limit  risk,  the  program 
may  not  have  gained  sufficient  knowledge  to 
enter  development.  Not  demonstrating  the 
maturity  of  the  design  before  critical  design  review 
perpetuates  this  risk  and  could  lead  to  poor  cost 
and  schedule  outcomes. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 

•  Demonstrate  all  critical  technologies  in  an  operational 
environment 

•  Complete  preliminary  design  review  o 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■*■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  CRH 


CRH  Program 

Technology  Maturity 

The  CRH  program  began  system  development  in 
June  2014  without  identifying  critical  technologies. 
In  transitioning  from  the  canceled  CSAR-X  program, 
the  Air  Force  reduced  risk  in  the  CRH  design  by 
lowering  requirements  and  replacing  the  immature 
technologies  identified  in  a  technology  readiness 
assessment  completed  for  CSAR-X  in  2006  with 
mature  subsystems.  The  CRH  program  plans  to 
modify  and  enhance  an  existing,  flight  proven 
helicopter-the  UH-60M-by  integrating  mature 
subsystems  and  associated  software.  The  planned 
modifications  and  enhancements  include  a  mission 
computer  and  software,  higher  capacity  electrical 
system,  larger  capacity  main  fuel  tanks,  armor  for 
crew  protection,  and  situational  awareness 
enhancements.  Based  on  this  design  proposal, 
DOD  authorized  the  program  to  bypass  technology 
development  and  enter  the  acquisition  process  at 
system  development.  DOD  also  waived  several 
activities  normally  required  to  obtain  program 
certification  prior  to  the  start  of  system 
development,  such  as  conducting  a  preliminary 
design  review,  demonstration  of  technology  in  a 
relevant  environment,  and  competitive  prototyping. 
In  2015,  the  program  conducted  systems 
engineering  reviews  and  reported  that  no  significant 
risks  or  issues  were  identified.  A  technology 
readiness  assessment  for  CRH  is  now  being 
conducted  by  Air  Force  personnel  independent  of 
the  program  office  in  advance  of  a  preliminary 
design  review  scheduled  for  April  2016.  Program 
officials  expect  this  assessment  to  confirm  or  rebut 
the  Air  Force's  previous  belief  that  the  program  has 
no  critical  technologies,  which  led  to  the  program 
waiver  decision. 

Design  and  Production  Maturity 

The  program  intends  to  monitor  CRH  design 
maturity  by  tracking  items,  such  as  the  completion 
and  approval  of  specifications,  the  number  of  design 
trades,  and  definitization  of  subcontract  awards. 
Program  officials  stated  that  early  prototype  testing 
to  demonstrate  the  design  will  not  be  needed  before 
the  critical  design  review,  currently  scheduled  for 
July  201 7,  as  CRH  is  a  derivative  of  the  operational 
HH-60M  helicopter.  However,  the  Air  Force  is 
integrating  major  new  subsystems  and  software  into 
this  platform,  and  several  risks  related  to  the 
integration  of  these  capabilities  must  be 


successfully  mitigated.  Currently  the  program  plans 
to  demonstrate  an  early  system  prototype  in 
October  2018,  15  months  after  the  scheduled 
assessment  of  the  design's  maturity  at  critical 
design  review.  Although  the  CRH  acquisition 
strategy  is  intended  to  ensure  the  incorporation  of 
mature  technologies  into  an  existing  aircraft,  by 
waiving  competitive  prototyping  and  the 
demonstration  of  technology  in  a  relevant 
environment,  and  forgoing  a  preliminary  design 
review  prior  to  the  start  of  system  development,  the 
program  may  not  have  gained  sufficient  knowledge 
to  enter  development  with  the  least  amount  of  risk. 
Not  demonstrating  the  maturity  of  the  design 
through  testing  with  a  fully  integrated  system-level 
prototype  before  critical  design  review  perpetuates 
this  risk.  In  our  previous  work,  we  have  found 
acquisition  programs  which  demonstrate  technology 
and  design  maturity  at  appropriate  points  typically 
have  better  cost  and  schedule  outcomes.  Officials 
claim  that  sufficient  amounts  of  development  testing 
are  planned  to  reduce  these  risks  before  the 
production  start  decision  in  2019,  including  the 
testing  of  a  production  representative  prototype. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  stated  that  an  independent  subject 
matter  expert  team  is  assessing  the  planned 
hardware  and  software  design  changes,  and  its  final 
assessment  is  planned  to  be  completed  in  January 
2016.  The  program  continually  tracks  risk  under  its 
risk  management  process.  Based  on  a  joint  risk 
management  board  held  in  November  2015,  the 
government  and  contractor  did  not  identify  any 
major  risks,  including  risks  related  to  technical 
maturity.  Mitigation  plans  have  been  prepared  for  all 
risks.  The  next  critical  program  milestone  is  the  air 
vehicle  preliminary  design  review  in  April  2016. 
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Common  Name:  EPS 


Enhanced  Polar  System  (EPS) 

The  Air  Force's  Enhanced  Polar  System  (EPS)  is  to 
provide  next-generation  protected  extremely  high 
frequency  (EHF)  satellite  communications  in  the 
polar  region.  It  will  replace  the  current  Interim  Polar 
System  and  serve  as  a  polar  adjunct  to  the 
Advanced  EHF  (AEHF)  system.  EPS  is  to  consist  of 
three  segments:  two  EHF  payloads  hosted  on 
classified  satellites,  a  Control  and  Planning 
Segment  (CAPS),  and  a  gateway  to  connect 
modified  Navy  Multiband  Terminals  to  other 
communication  systems. 


Source:  LinQuest  Corporation. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Northrop 

As  of 

Latest 

Percent 

Grumman 

04/2014 

12/2014 

change 

Program  office:  El  Segundo,  CA 

Research  and  development  cost 

$1,431.0 

$1,424.7 

-0.4% 

Funding  needed  to  complete: 

Procurement  cost 

$0.0 

$0.0 

0.0% 

R&D:  $144.4  million 

Total  program  cost 

$1,431.0 

$1,424.7 

-0.4% 

Procurement:  $0.0  million 

Program  unit  cost 

$715,515 

$712,373 

-0.4% 

Total  funding:  $144.4  million 

Total  quantities 

2 

2 

0.0% 

Procurement  quantity:  0 

Acquisition  cycle  time  (months) 

126 

126 

0.0% 

The  EPS  program  formally  entered  system 
development  in  April  2014.The  three  segments  of 
EPS-payloads,  CAPS,  and  the  gateway-are  in 
various  stages  of  completion.  Although  the  EPS 
program's  two  EHF  payloads  are  built  and  the  first 
is  on-orbit,  funding  constraints  resulted  in 
redesign  and  delays  to  CAPS  and  the  gateway 
segments,  in  turn  delaying  initial  operational 
capability  by  2  years.  CAPS  completed  software 
development  in  October  2015,  and  is  proceeding 
with  integration  and  testing.  The  gateway  site 
installation  is  complete  and  testing  was 
completed  in  December  2015.  System  inter¬ 
segment  testing,  which  will  test  all  three  elements 
together,  is  expected  to  be  completed  in  August 
2016.  Operational  testing  is  scheduled  for 
completion  in  2018  and  is  the  last  significant 
schedule  event,  as  there  are  no  production 
related  decisions  for  the  program. 


Attainment  of  Product  Knowledge 
As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  ^ 

environment  ^ 

•  Demonstrate  all  critical  technologies  in  an  operational  a 

environment  w 

•  Complete  preliminary  design  review  £ 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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EPS  Program 

Technology  and  Design  Maturity 

The  EPS  program  formally  entered  system 
development  in  April  2014.  The  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics 
directed  the  program  to  proceed  to  system 
development  to  synchronize  the  program's  payload 
development  schedule  with  the  host  satellite's 
production  timeline.  The  program  office  for  the  host 
satellite  awarded  the  payload  development  contract 
in  July  2008,  following  an  acquisition  board 
equivalent  to  a  system  development  decision.  All 
three  segments  of  the  program  will  be  completed 
under  a  development  effort,  and  there  are  no 
production-related  decisions  for  this  program. 

Both  payloads  are  built.  The  first  payload  became 
available  for  on-orbit  testing,  which  includes 
hardware  functions  and  uplink  and  downlink 
capability,  in  March  2015.  The  second  payload  is 
integrated  into  the  host  satellite,  is  undergoing 
system-level  testing,  and  is  expected  to  be  on  orbit 
in  the  last  quarter  of  fiscal  year  201 7.  According  to 
program  officials,  all  EPS  hardware  development 
and  critical  technologies  are  associated  with  the 
payloads. 

In  contrast  to  payload  development,  the  CAPS  and 
the  gateway  segment  were  delayed  as  the  program 
office  used  a  design-to-cost  approach  to  reduce 
CAPS  and  gateway  requirements  to  the  minimum 
capability  needed  due  to  funding  constraints.  The 
revised  EPS  acquisition  strategy  incorporating 
changes  to  CAPS  and  the  gateway  segment  was 
approved  in  January  2012,  and  the  program  was 
approved  to  enter  system  development  in  April 
2014. 

The  only  development  work  remaining  in  the 
program  is  with  CAPS.  According  to  program 
officials,  CAPS  is  primarily  a  software  development 
effort  and  utilizes  commercial  off-the-shelf 
hardware.  As  such,  the  program  uses  software- 
related  metrics,  including  software  lines  of  code 
delivered,  to  track  development  progress.  According 
to  the  program  office,  CAPS  software  development 
was  completed  in  October  2015  and  is  in  the 
process  of  software  integration  and  testing.  The 
gateway  primarily  involves  integration  of  existing 
equipment  and  is  considered  low  risk  by  the 
program  office.  Integration  includes  commercial  off- 


the-shelf  hardware  such  as  routing  and  switching 
equipment,  and  terminals  developed  under  other 
programs.  The  Navy's  Space  and  Naval  Warfare 
Systems  Command  Systems  Center  Pacific  and 
MIT/Lincoln  Laboratory  are  responsible  for 
integrating,  testing,  and  installing  the  gateway 
segment.  Site  installation  for  the  gateway  is 
complete,  and  the  program  office  expects 
verification  testing  will  be  completed  by  the  end  of 
2015.  System  level  inter-segment  interface  testing 
of  payload,  gateway,  and  CAPS  is  scheduled  for 
completion  in  August  2016. 

Other  Program  Issues 

In  its  comments  on  a  draft  of  this  assessment,  the 
program  office  stated  that,  the  2012  revision  of  the 
acquisition  strategy  reduced  program  risk  by 
reducing  requirements  for  additional  development. 
This  decreased  overall  program  cost  by  about  $1 
billion.  However,  the  changes  also  delayed  initial 
operational  capability  from  fiscal  year  201 6  to  201 8. 
According  to  program  officials,  the  delay  is  not 
expected  to  result  in  a  gap  in  coverage. 

Program  Office  Comments 

The  EPS  program  officials  provided  technical 
comments,  which  were  incorporated  as  appropriate. 
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Common  Name:  EELV 


Evolved  Expendable  Launch  Vehicle  (EELV) 

The  Air  Force's  EELV  program  provides  spacelift 
support  for  DOD,  national  security,  and  other 
government  missions.  While  the  United  Launch 
Alliance  (ULA)  is  the  only  provider  of  launch 
vehicles  and  support  functions  currently  under 
contract,  Space  Exploration  Technologies  (SpaceX) 
was  certified  to  compete  for  national  security 
launches  with  its  Falcon  9  launch  vehicle  in  May 
2015.  ULA  provides  launch  services  for  EELV  using 
two  families  of  launch  vehicles:  Atlas  V  and  Delta  IV. 

We  assessed  the  vehicle  variants  from  both  ULA 
and  SpaceX. 

Source:  ©2012  United  Launch  Alliance. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  United  Launch 

As  of 

Latest 

Percent 

Services,  LLC 

10/1998 

10/2014 

change 

Program  office:  El  Segundo,  CA 

Research  and  development  cost 

$1,910.2 

$3,008.0 

57.5% 

Funding  needed  to  complete: 

Procurement  cost 

$16,731.9 

$57,489.0 

243.6% 

R&D:  $279.9  million 

Total  program  cost 

$18,642.1 

$60,496.9 

224.5% 

Procurement:  $34,734.2  million 

Program  unit  cost 

$102,995 

$366,648 

256.0% 

Total  funding:  $35,014.1  million 

Total  quantities 

181 

165 

-8.8% 

Procurement  quantity:  88 

Acquisition  cycle  time  (months) 

N/A 

120 

N/A 

We  assessed  EELV  technology  as  mature  with  90 
successful  launches  as  of  December  2015.  EELV 
design  and  production  maturity  are  not  assessed 
using  our  best  practices,  but  using  an  Aerospace 
Corporation  measure  that  was  developed  for  the 
program.  Using  that  measure,  eleven  of  the  14 
EELV  launch  vehicle  variants  offered  by  ULA 
have  demonstrated  design  maturity,  as  has 
SpaceX's  Falcon  9.  EELV  is  currently  assessing 
options  for  an  alternative  to  the  Russian-made 
RD-180  engine  used  on  the  Atlas  V  vehicle,  with 
industry  expressing  support  for  government 
investment  in  technology  development  and 
shared  investment  in  launch  service 
development,  including  rocket  propulsion  system 
development  efforts.  The  program  is  also 
exploring  options  for  an  alternative  to  the  Delta  IV 
Medium  launch  vehicle,  which  ULA  plans  to 
discontinue  in  the  fiscal  year  2018  to  2019 
timeframe. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


•  Release  at  least  90  percent  of  design  drawings 


•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

9  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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EELV  Program 

Technology,  Design,  and  Production  Maturity 

Each  of  the  EELV  14  variants  offered  by  ULA,  and 
the  Falcon  9  from  SpaceX,  have  flown  at  least  once, 
demonstrating  the  viability  of  their  technologies.  For 
design  and  production  maturity,  launch  vehicles  are 
assessed  using  the  3/7  reliability  rule  developed  by 
the  Aerospace  Corporation.  According  to  this  rule, 
once  a  variant  is  launched  successfully  three  times, 
its  design  can  be  considered  mature.  Similarly,  if  a 
variant  is  successfully  launched  seven  times,  both 
the  design  and  production  process  can  be 
considered  mature.  Eleven  of  the  ULA  variants  have 
achieved  design  maturity,  and  three  have  reached 
both  design  and  production  maturity.  Some  variants 
are  used  infrequently  and  may  never  reach  design 
or  production  maturity.  The  Falcon  9  vl.1  has 
achieved  both  design  and  production  maturity,  but 
Air  Force  officials  said  SpaceX  intends  to  use  a  new 
variant-the  Falcon  9  Upgrade-for  future  EELV 
launch  service  competitions.  This  variant  was 
successfully  flown  for  the  first  time  in  December 
2015.  New  variants  introduce  changes  to  the 
original  design,  which,  until  proven  through  multiple 
successful  flights,  pose  potential  cost  and  schedule 
risks. 

In  August  2014,  the  Air  Force  released  a  request  for 
information  on  development  of  a  domestic  booster 
and  launch  system  to  replace  the  RD-180.  It 
subsequently  reported  that  industry  responses 
indicate  support  both  for  investment  in  technical 
maturation  at  the  propulsion  system  level  as  well  as 
shared  investment  in  development  efforts  with 
launch  providers  and  competition  for  launch 
services-much  like  the  original  EELV  program  and 
the  NASA  Commercial  Orbital  Transportation 
Services,  Cargo,  and  Commercial  Crew  programs. 

Other  Program  Issues 

ULA  plans  to  phase  out  its  Delta  IV  medium-lift 
class  of  vehicles  in  the  fiscal  year  2018  to  2019 
timeframe  and  will  introduce  its  new  Vulcan  launch 
vehicle  currently  in  development  with  an  initial 
capability  expected  in  2019.  Unless  a  replacement 
for  the  RD-180  and  Atlas  V  capability  can  be 
developed  on  this  very  aggressive  timeline,  this 
could  result  in  a  capability  gap  for  some  national 
security  space  launch  missions  in  fiscal  years  2020- 
2021 ,  as  the  Falcon  9  does  not  have  the  capacity  to 
launch  all  national  security  payloads.  ULA  has 


agreed  to  maintain  the  Delta  IV  heavy-lift  class  to 
meet  national  security  space  requirements  until  a 
replacement  system  is  available.  The  Air  Force  and 
ULA  are  exploring  continued  use  of  the  Delta  IV 
medium-lift  class.  For  example,  they  are  looking  at 
the  possibility  of  preserving  Delta  IV  tooling  and 
expertise  to  reconstitute  the  production  line,  if 
needed. 

According  to  Air  Force  officials,  one  possible 
remedy  would  be  availability  of  SpaceX's  proposed 
Falcon  Heavy  launch  vehicle,  which  should  be 
capable  of  meeting  payload  requirements  in  the 
EELV  intermediate  and  heavy  classes,  currently 
being  met  by  Atlas  V  and  Delta  IV  Heavy.  However, 
Air  Force  officials  said  SpaceX  has  not  yet 
demonstrated  the  Falcon  Heavy  in  flight,  does  not 
expect  to  do  so  until  early  2016,  and  has  only 
recently  begun  working  with  the  Air  Force  toward 
vehicle  certification. 

In  September  2015,  the  Air  Force  issued  a  request 
for  proposal  for  EELV  launch  services,  the  first  since 
SpaceX's  Falcon  9  certification.  Following  the 
request,  ULA  announced  that  it  did  not  intend  to  bid 
for  the  launch  to  put  the  second  GPS  III  satellite  into 
orbit,  stating,  among  other  things,  that  prohibitions 
on  the  use  of  the  RD-180  restrict  the  company's  use 
of  the  Atlas  V.  The  National  Defense  Authorization 
Act  (NDAA)  for  fiscal  year  2015  prohibited,  with 
certain  exceptions,  the  award  or  renewal  of  a 
contract  for  the  procurement  of  property  or  services 
for  space  launch  activities  under  the  EELV  program 
if  such  contract  carries  out  such  activities  using 
rocket  engines  designed  or  manufactured  in  the 
Russian  Federation.  The  NDAA  for  fiscal  year  201 6 
amended  the  exceptions  to  the  prohibition.  The 
201 6  DOD  Appropriation  Act  stated  that 
notwithstanding  any  other  provision  of  law,  award 
can  be  made  to  a  launch  service  provider 
competing  with  any  certified  launch  vehicle  in  its 
inventory  regardless  of  the  origin  of  the  rocket 
engine. 

Program  Office  Comments 

The  Air  Force  provided  technical  comments,  which 
were  incorporated  as  appropriate. 


Page  136 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


Lead  Component:  Air  Force 
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F-22  Increment  3.2B  Modernization  (F-22  Inc  3.2B  Mod) 


The  Air  Force's  F-22  Raptor  is  a  stealthy  air-to-air 
and  air-to-ground  fighter/attack  aircraft.  The  Air 
Force  established  an  F-22  modernization  and 
improvement  program  in  2003  to  add  enhanced  air- 
to-ground,  information  warfare,  reconnaissance, 
and  other  capabilities,  and  to  improve  the  reliability 
and  maintainability  of  the  aircraft.  Increment  3.2B, 
the  fourth  increment  of  the  modernization  program 
was  initially  managed  as  part  of  the  F-22  baseline 
program,  but  is  now  managed  as  a  separate  major 
defense  acquisition  program. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Lockheed  Martin 

As  of 

Latest 

Percent 

Program  office:  Wright-Patterson  AFB, 

06/2013 

07/2015 

change 

OH 

Research  and  development  cost 

$1,254.2 

$1,218.2 

-2.9% 

Funding  needed  to  complete: 

Procurement  cost 

$353.7 

$347.1 

-1 .9% 

R&D:  $214.9  million 

Total  program  cost 

$1,607.9 

$1,565.3 

-2.6% 

Procurement:  $319.1  million 

Program  unit  cost 

$10,578 

$10,298 

-2.6% 

Total  funding:  $534.0  million 

Total  quantities 

152 

152 

0.0% 

Procurement  quantity:  143 

Acquisition  cycle  time  (months) 

72 

75 

4.2% 

Increment  3.2B  completed  its  design  review  in 
October  2015,  after  a  3  month  delay,  with  its  one 
critical  technology  fully  mature  and  its  design 
stable  and  demonstrated  with  98  percent  of  its 
drawings  releasable.  The  one  reported  critical 
technology  has  been  demonstrated  in  a  realistic 
environment,  and,  according  to  program  officials, 
flight  testing  will  conclude  in  December  2015.  The 
program  is  using  an  iterative  software 
development  process  with  seven  releases  of 
capability,  up  one  from  initial  estimates.  Further 
delays  in  fielding  earlier  F-22  modernization 
increments  could  have  an  effect  on  fielding  of 
Increment  3.2B,  as  the  increments  build  upon 
each  other.  Increment  3.2B  received  approval  to 
begin  system  development  as  a  separate  major 
defense  acquisition  program  in  June  2013. 
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Common  Name:  F-22  Inc  3.2B  Mod 


F-22  Inc  3.2B  Mod  Program 

Technology  and  Design  Maturity 

The  program's  sole  identified  critical  technology,  a 
geolocation  algorithm,  is  fully  mature.  It  has  been 
demonstrated  in  a  realistic  environment,  and, 
according  to  program  officials,  will  complete  flight 
testing  in  December  2015.  The  program 
demonstrated  its  design  prior  to  the  October  2015 
critical  design  review  and  98  percent  of  system-level 
drawings  are  currently  releasable.  According  to 
officials,  the  program  intends  to  conduct 
demonstrations  with  an  integrated,  system-level 
prototype  through  December  2015.  The  critical 
design  review  was  a  culmination  of  multiple 
incremental  critical  design  reviews,  with  the  October 
2015  review  focused  on  software  as  the  program 
had  already  completed  hardware  reviews. 

According  to  a  program  official,  initial  flight  testing  of 
the  software  has  already  begun.  The  program  is 
using  an  iterative  software  development  process 
where  six  software  releases  had  been  planned.  Due 
to  interface  issues,  the  program  added  an  additional 
software  release,  for  a  total  of  seven.  This  software 
release  is  only  for  lab  testing  but  added  an 
additional  month  to  the  program's  schedule  as  well 
as  some  cost,  according  to  officials. 

Other  Program  Issues 

Program  officials  stated  that  all  F-22  Increment  3.2B 
modifications  will  be  completed  by  contractor  field 
teams  at  bases  and  not  in  the  depot,  as  this  is  a  less 
costly  approach.  Depot-level  maintenance  refers  to 
major  maintenance  and  repairs,  such  as 
overhauling,  upgrading,  or  rebuilding  parts, 
assemblies,  or  subassemblies,  which  is  usually 
performed  at  a  facility  known  as  a  depot.  Program 
officials  noted  that  the  schedule  for  fielding 
Increment  3.1  has  been  affected  due  to  depot-level 
delays.  Increment  3.2A  is  a  software  upgrade  that 
requires  Increment  3.1  hardware,  and  delays  in  the 
Increment  3.1  schedule  have  slowed  the  expected 
fielding  of  Increment  3.2A.  Further  delays  may 
affect  the  time  line  for  fielding  future  modernization 
increments,  as  the  increments  build  on  each  other. 
Any  further  delays  in  fielding  these  earlier 
increments  could  affect  fielding  of  Increment  3.2B. 


Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  stated  that  in  April  2015,  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics  re-designated  the  F-22  Increment 
3.2B  program  as  an  ACAT  1C  program  and 
delegated  milestone  decision  authority  to  the  Air 
Force.  To  date,  the  Increment  3.2B  program  is 
proceeding  as  planned.  Software  lab  testing  was 
completed  on  the  seventh  agile  software 
development  release  iteration  in  September  2015. 
The  program  also  completed  software  lab  testing  on 
the  first  fully  integrated  developmental  test  software 
version  (FI)  and  is  conducting  FI  developmental 
test  flight  test  operations  using  the  five  Increment 
3.2B  modified  flight  test  aircraft.  The  remaining  four 
test  aircraft  to  receive  Increment  3.2B  hardware 
modifications  are  on  schedule  to  be  completed  by 
May  2016.  In  preparation  for  a  low-rate  production 
decision,  the  program  is  completing  its 
manufacturing  readiness  assessment  and  has 
assessed  the  prime  contractor  and  the  suppliers 
manufacturing  Increment  3.2B  end  item  hardware. 
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Lead  Component:  Air  Force 


Common  Name:  FAB-T 


Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T) 


The  Air  Force's  FAB-T  program  plans  to  provide  a 
family  of  satellite  communications  terminals  for 
airborne  and  ground-based  users  to  replace  many 
program-unique  terminals.  Designed  to  work  with 
current  and  future  communications  capabilities  and 
technologies,  FAB-T  is  expected  to  provide  voice 
and  data  communications  over  military  satellite 
networks  for  nuclear  and  conventional  forces 
through  ground  command  posts  and  E-6  and  E-4 
aircraft.  A  subprogram  is  expected  to  provide  force 
element  capabilities  on  B-2,  B-52,  and  RC-135 
aircraft. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Raytheon 

As  of 

Latest 

Percent 

Program  office:  Bedford,  MA 

12/2006 

12/2014 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$1,649.1 

$2,718.9 

64.9% 

R&D:  $218.8  million 

Procurement  cost 

$1,771.8 

$1,589.3 

-10.3% 

Procurement:  $1,378.6  million 

Total  program  cost 

$3,420.9 

$4,308.3 

25.9% 

Total  funding:  $1,597.4  million 

Program  unit  cost 

$15,837 

$16,634 

19.9% 

Procurement  quantity:  189 

Total  quantities 

216 

259 

19.9% 

Acquisition  cycle  time  (months) 

129 

207 

60.5% 

The  latest  cost  and  quantity  data  do  not  reflect  recent  program  decisions.  A  new  acquisition  program 

baseline  has  not  yet  been  approved. 

While  the  technology  and  design  for  10  initial 
FAB-T  units  appear  to  be  stable,  testing  remains 
for  final  configurations  using  new  antennae  and 
currently  immature  technologies.  In  September 
2015,  the  Air  Force  received  approval  to  begin 
low-rate  production  and  expects  to  mature 
production  processes  after  factory  testing  of  initial 
units.  However,  the  initial  units  will  require 
retrofitting  with  new  antennae.  The  program  has 
not  yet  planned  for  developing  and  producing 
force  element  terminals  needed  to  achieve 
FAB-T's  full  communications  capabilities,  but  the 
Air  Force  expects  to  provide  a  plan  by  June  201 6. 
Program  delays  to  FAB-T  prevent  the  full 
utilization  of  Advanced  Extremely  High  Frequency 
Satellite  capabilities.  The  Air  Force  expected 
approval  of  a  revised  cost  and  schedule  baseline 
by  the  end  of  201 5,  but  it  has  not  yet  been 
approved. 
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Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 


o 


Demonstrate  all  critical  technologies  in  an  operational 
environment 


o 


Complete  preliminary  design  review 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 
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9  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 

Not  applicable 


Common  Name:  FAB-T 


FAB-T  Program 

Technology  and  Design  Maturity 

According  to  the  program  office,  the  technologies 
and  design  used  in  the  configuration  for  the  first  lot 
of  low-rate  production  units  are  mature.  However, 
this  initial  lot  does  not  reflect  the  final  configuration 
of  the  terminal,  which  will  use  a  new  antenna,  and 
these  initial  units,  while  fully  functional,  will 
eventually  need  to  be  retrofitted.  The  new  antenna 
includes  five  separate  critical  technologies,  four  of 
which  are  currently  immature.  Program  officials 
estimated  that  the  new  antenna  technologies  will  be 
mature  by  September  2016.  The  design  for  the  final 
configuration  is  stable,  and  program  officials  said 
they  expect  only  minor  revisions  or  updates  to  the 
existing  design  as  they  complete  testing. 

Production  Maturity 

FAB-T  received  verbal  approval  to  begin  low-rate 
production  in  September  2015,  followed  by  a  formal 
acquisition  decision  in  October  2015.  The  program 
expects  to  fully  mature  its  production  process 
following  production  and  factory  testing  of  its  initial 
low-rate  units.  The  program  intends  to  purchase 
more  than  60  percent  of  its  total  units  during  low- 
rate  production.  Generally,  programs  must  provide  a 
rationale  if  low-rate  production  quantities  are  going 
to  exceed  10  percent  of  total  quantities.  Program 
officials  explained  that  these  units  are  required  to 
demonstrate  initial  operational  capability  by  the  end 
of  2019. 

Other  Program  Issues 

In  2012,  FAB-T's  acquisition  strategy  was  changed 
to  offer  two  possible  production  paths-one  providing 
both  command  post  and  force  element  terminals 
and  the  other  only  command  post  terminals.  In  July 
2015,  the  Department  of  Defense  separated  the 
command  post  terminals  and  force  element 
terminals  into  two  sub-programs.  Currently,  only  the 
command  post  terminals  subprogram  is  in 
development  and  production,  while  the  force 
element  terminals  are  not  funded  in  the  2016 
President's  budget  submission.  If  not  integrated  with 
the  B-2  and  B-52  bomber  platforms,  FAB-T  may  not 
achieve  its  planned  capabilities  as  some  are  based 
on  the  interaction  of  bomber  aircraft  with 
intelligence,  surveillance,  and  reconnaissance 
aircraft  and  command  post  terminals.  In  October 
2015,  the  Undersecretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  instructed  the 


Air  Force  to  develop  a  strategy  for  achieving  the 
force  element  terminal  requirements  by  February 
2016,  and  the  Air  Force  requested  an  extension 
until  June  2016. 

FAB-T  plans  to  communicate  through  the  Advanced 
Extremely  High  Frequency  network  of  satellites. 
Three  of  these  satellites  have  already  been 
launched,  but  FAB-T  has  yet  to  be  fielded,  resulting 
in  underutilization  of  these  costly  satellite 
capabilities.  By  the  time  FAB-T  achieves  initial 
operating  capability  in  2019,  one  of  these  satellites 
will  have  been  operating  for  nine  years,  over  half  of 
its  projected  operational  lifetime. 

The  Air  Force  expected  approval  of  a  revised  cost 
and  schedule  baseline  by  the  end  of  201 5,  based  on 
a  new  independent  cost  estimate;  however,  it  has 
not  yet  been  approved.  The  new  baseline  will  reflect 
the  numerous  program  changes  to  the  current 
baseline  established  in  2007. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  stated  that  in  September  2015,  the 
program  conducted  a  successful  production 
decision,  resulting  in  verbal  approval  by  the 
milestone  decision  authority  to  purchase  the  first  10 
of  53  low-rate  initial  production  terminals. 
Production  is  underway  with  the  first  terminal 
delivery  anticipated  in  December  2016. 
Development  and  testing  is  ongoing  for  the 
remaining  antenna  configurations  to  demonstrate 
maturity  prior  to  commencing  production  of  these 
configurations.  The  projected  start  of  production  of 
the  final  configurations  was  the  third  quarter  of  fiscal 
year  2016;  however,  the  program  office  is  now 
looking  at  an  incremental  release  of  new  antenna 
configurations  beginning  in  the  fourth  quarter  of 
fiscal  year  2016,  with  the  final  configuration 
beginning  in  the  first  quarter  of  fiscal  year  2017.  The 
program  is  on  track  to  meet  initial  operational 
capability  in  December  2019. 
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Lead  Component:  Air  Force 


Common  Name:  GPS  III 


Global  Positioning  System  III  (GPS  III) 

The  Air  Force's  Global  Positioning  System  (GPS)  III 
program  plans  to  develop  and  field  a  new 
generation  of  satellites  to  supplement  and 
eventually  replace  GPS  satellites  currently  in  use. 

Other  programs  are  developing  the  related  ground 
system  and  user  equipment.  GPS  III  is  intended  to 
provide  capabilities  for  a  stronger  military  navigation 
signal  to  improve  jamming  resistance  and  a  new 
civilian  signal  that  will  be  interoperable  with  foreign 
satellite  navigation  systems. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Lockheed  Martin 

As  of 

Latest 

Percent 

Program  office:  El  Segundo,  CA 

05/2008 

07/2015 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$2,708.5 

$3,180.1 

17.4% 

R&D:  $224.4  million 

Procurement  cost 

$1,520.7 

$1,737.4 

14.3% 

Procurement:  $235.7  million 

Total  program  cost 

$4,229.2 

$4,917.5 

16.3% 

Total  funding:  $460.1  million 

Program  unit  cost 

$528,644 

$614,682 

16.3% 

Procurement  quantity:  0 

Total  quantities 

8 

8 

0.0% 

Acquisition  cycle  time  (months) 

N/A 

N/A 

N/A 

We  could  not  calculate  acquisition  cycle  times  for  the  first  increment  of  the  GPS  III  program  because 

initial  operational  capability  will  not  occur  until  satellites  from  a  future  increment  are  fieldec 

The  program  currently  reports  all  technologies  as 
mature  and  the  design  as  stable  but 
manufacturing  processes  as  not  yet  in  control. 
The  first  GPS  III  satellite  completed  baseline 
thermal  vacuum  testing-a  major  system-level 
event  for  the  program.  GPS  III  is  rebaselining  its 
cost  and  schedule  estimates  due  to  delays  in  the 
availability  for  launch  first  reported  in  early  2014. 
Program  officials  stated  that  the  new  available  for 
launch  date  is  August  2016-almost  two  and  a  half 
years  later  than  originally  planned.  These  delays 
are  accompanied  by  even  more  extensive  delays 
on  the  related  ground  system-the  Next 
Generation  Operational  Control  System  (OCX)- 
and  even  later  delivery  of  military  GPS  user 
equipment  program.  Both  are  needed  to  provide 
all  GPS  III  planned  capabilities. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 

•  Demonstrate  all  critical  technologies  in  an  operational 
environment 

•  Complete  preliminary  design  review 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

• 

•  Test  a  system-level  integrated  prototype 

• 

1  Manufacturing  processes  are  mature 

•  Demonstrate  critical  processes  are  in  control 

o 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

•  Test  a  production-representative  prototype 

o 

9  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

O  Knowledge  not  attained  Not  applicable 
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Common  Name:  GPS  III 


GPS  III  Program 

Technology  Maturity 

The  program  office  reports  that  all  eight  GPS  III 
critical  technologies  are  fully  mature.  The  program 
previously  experienced  design  and  manufacturing 
problems  with  the  mission  data  unit,  a  key 
component  of  the  navigation  payload,  which 
delayed  the  program  by  almost  2  years.  The 
program  office  reported  that  those  problems  have 
been  resolved  and  that  verification  of  its  design  and 
manufacturing  processes  are  complete. 

Design  and  Production  Maturity 

The  program  office  reports  that  the  GPS  III  design  is 
currently  stable  based  on  the  number  of  design 
drawings  released  to  manufacturing.  To  prove  out 
production  processes  prior  to  integrating  and  testing 
the  first  space  vehicle,  the  program  tested  a 
system-level  integrated  prototype  that  included  all 
key  subsystems  and  components,  but  less 
redundancy  than  the  final  configuration.  A  complete 
satellite  was  not  tested  prior  to  GPS  Ill's  production 
decision  in  2011,  but  the  program  reported  a  level  of 
manufacturing  process  maturity  that  indicated  its 
processes  were  in  control  and  production  could 
begin.  However,  the  program  recently  reported  a 
lower  level  of  maturity  that  indicates  production 
processes  are  not  yet  fully  in  control.  This  level  of 
maturity  is  high  enough  to  meet  DOD's  standards 
for  the  start  of  production,  but  not  those 
recommended  by  best  practices. 

Other  Program  Issues 

In  December  2015,  the  first  GPS  III  space  vehicle 
completed  baseline  thermal  vacuum  testing.  This 
key  test  event  validates  satellite  performance  in  a 
simulated  space  environment  in  order  to 
demonstrate  the  satellite's  performance  under 
environmental  extremes  and  to  increase  mission 
assurance. 

Due  to  a  delay  to  launch  availability  of  over  2  years 
first  reported  in  early  2014,  the  program  is  in  the 
process  of  rebaselining  costs  and  schedule. 
Program  officials  stated  that  the  new  available  for 
launch  date  is  August  2016-seven  months  later 
than  GAO  previously  reported  and  almost  two  and  a 
half  years  later  than  originally  planned.  Total 
acquisition  costs  in  the  new  baseline  will  likely  be 


higher  and  the  quantity  of  satellites  will  change  from 
8  to  1 0.  Approval  of  the  new  baseline  is  expected  in 
2016. 

Although  elements  of  the  GPS  III  program,  such  as 
thermal  vacuum  testing,  have  experienced  delays, 
the  satellites'  dependence  on  the  much  delayed 
OCX  is  a  greater  risk  to  constellation  sustainment. 
GPS  III  satellites  cannot  be  integrated  into  the 
constellation  or  achieve  operational  capability  until 
OCX  Block  1  is  delivered,  currently  scheduled  for 
July  2021 , 6  years  later  than  originally  planned.  As  a 
result  of  these  delays,  several  GPS  III  satellites  may 
be  available  for  launch  before  they  are  fully  tested 
with  OCX.  However,  the  Air  Force  reported  that  it  is 
taking  steps  to  enable  GPS  III  integration  into  the 
constellation  by  spring  2019.  Moreover,  although 
testing  the  satellites  functionality  is  not  dependent 
on  new  military  GPS  user  equipment,  timing  of 
capability  delivery  for  that  program  will  further 
postpone-by  about  a  decade-the  warfighters  ability 
to  take  advantage  of  newer  GPS  III  satellite 
capabilities,  such  as  improved  resistance  to 
jamming. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  stated  that  the  first  GPS  III  satellite  is 
on  track  for  an  August  2016  available  for  launch 
date,  and  that  significant  confidence  was  gained  in 
the  contractor  design  and  workmanship  as  a  result 
of  thermal  vacuum  testing.  The  program  office  also 
noted  that  the  contractor  demonstrated  production 
line  maturity  and  that  vigilant  monitoring  for  future 
problems  will  continue  to  mitigate  delays.  The  next 
major  test  event  for  the  first  space  vehicle  will  be 
electromagnetic  interference  testing  in  March  2016. 
Technical  comments  were  provided  by  the  program 
office,  which  were  incorporated  where  deemed 
appropriate. 
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Lead  Component:  Air  Force 


Common  Name:  KC-46A 


KC-46  Tanker  Modernization  Program  (KC-46A) 

The  Air  Force's  KC-46  program  plans  to  convert  a 
commercial  aircraft  into  an  aerial  refueling  tanker  for 
operations  with  Air  Force,  Navy,  Marine  Corps,  and 
allied  aircraft.  The  program  is  the  first  of  three 
planned  phases  to  replace  the  Air  Force's  aging 
aerial  refueling  tanker  fleet.  The  KC-46  has  been 
designed  to  improve  on  the  KC-135's  refueling 
capacity,  efficiency,  capabilities  for  cargo  and 
aeromedical  evacuation,  and  to  integrate  defensive 
systems. 


Source:  ©  2011  Boeing.  All  rights  reserved. 


System  development 


Production 


▲  ▲ 

Development  Critical 

start  design  review 

(2/11)  (7/13) 

▲  ▲  ▲ 

KC-46  GAO  Low-rate 

first  flight  review  decision 

(9/15)  (1/16)  (4/16) 

▲ 

Start 

operational  test 
(4/17) 

▲ 

Required  assets 
available 
(8/17) 

A 

Full-rate 

decision 

(3/18) 

Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Boeing 

Program  office:  Wright-Patterson  AFB, 

OH 

Research  and  development  cost 

As  of 
02/2011 

$7,351.9 

Latest 

07/2015 

$6,757.3 

Percent 

change 

-8.1% 

Funding  needed  to  complete: 

Procurement  cost 

$35,699.9 

$33,857.3 

-5.2% 

R&D:  $924.3  million 

Total  program  cost 

$47,021.3 

$43,531.6 

-7.4% 

Procurement:  $32,317.9  million 

Program  unit  cost 

$262,689 

$243,193 

-7.4% 

Total  funding:  $35,718.7  million 

Total  quantities 

179 

179 

0.0% 

Procurement  quantity:  168 

Acquisition  cycle  time  (months) 

78 

78 

0.0% 

One  of  KC-46's  three  critical  technologies  is  fully 
mature,  and  the  other  two  are  expected  to  be  fully 
mature  prior  to  the  production  decision  in  2016. 
The  design  was  considered  stable  at  the  July 
2013  critical  design  review,  but  wiring  and  fuel 
system  design  changes  were  subsequently 
made.  Demonstration  of  a  system-level 
prototype-KC-46  first  flight-was  achieved  in 
September  2015,  more  than  2  years  after  critical 
design  review.  Boeing  is  building  four 
development  aircraft  and  has  begun  fabricating 
additional  production  units.  A  key  supplier  is  not 
delivering  aerial  refueling  systems  on  time,  which 
could  affect  the  timely  delivery  of  aircraft. 
According  to  the  program  office,  it  will  continue  to 
monitor  and  assess  the  maturity  of  production 
following  Department  of  Defense  guidance. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  KC-46A 


KC-46A  Program 

Technology  Maturity 

The  program  office  identified  three  critical 
technologies  for  KC-46,  two  software  modules 
related  to  situational  awareness  and  a  three- 
dimensional  display  that  allows  the  crew  to  monitor 
aerial  refueling  activities.  Currently,  one  of  these 
three  critical  technologies-a  situational  awareness 
software  module-is  fully  mature.  The  other  two 
technologies  are  nearing  full  maturity,  having  been 
tested  in  a  laboratory.  The  program  plans  to  fully 
mature  all  technologies  through  laboratory  and  flight 
testing,  prior  to  the  low-rate  production  decision 
currently  scheduled  for  April  2016. 

Design  Maturity 

At  its  critical  design  review,  the  program  had 
released  over  90  percent  of  its  design  drawings. 
Since  that  time,  problems  were  discovered  with  the 
wiring  of  the  commercial  aircraft  and  the  design  of 
the  aerial  refueling  system  that  led  to  revisions. 
Boeing  is  behind  schedule  in  building  the  first  four 
aircraft  due  to  this  redesign  and  late  deliveries  of 
components.  A  key  supplier  has  experienced 
significant  difficulty  in  manufacturing  and  completing 
qualification  testing  of  the  drogue  refueling  systems 
due,  in  part,  to  challenges  complying  with  Federal 
Aviation  Administration  component  qualification 
procedures.  This  adds  risk  to  Boeing's  ability  to 
deliver  low-rate  production  aircraft  in  August  2017. 

As  a  result  of  these  delays,  first  flight  of  an 
integrated  KC-46  aircraft  with  military  sub-systems 
to  demonstrate  design  maturity  slipped  more  than  8 
months  to  late  September  2015,  about  26  months 
after  critical  design  review. 

Production  Maturity 

The  program  office  and  Boeing  have  taken  several 
initial  steps  to  capture  necessary  manufacturing 
knowledge.  As  of  August  2015,  the  program  office 
has  completed  manufacturing  readiness 
assessments  on  all  critical  manufacturing 
processes.  According  to  the  program  office,  it  will 
continue  to  monitor  and  assess  the  maturity  of 
production  following  standards  set  in  Department  of 
Defense  guidance.  Due  to  redesigns,  manufacturing 
processes  are  being  conducted  out  of  sequence  on 
the  four  development  aircraft  currently  in  fabrication. 


Other  Program  Issues 

Boeing  has  experienced  test  failures  with  aerial 
refueling  system  components  and  test  related 
delays,  such  as  an  accidental  fuel  contamination 
event  that  damaged  the  second  development 
aircraft's  fuel  system.  Boeing  is  re-baselining  its 
schedule  to  account  for  these  delays  and  has  been 
working  with  the  program  office  to  restructure  the 
program.  The  low-rate  initial  production  decision  is 
currently  scheduled  for  April  2016.  Boeing  plans  to 
complete  roughly  60  hours  of  aerial  refueling 
demonstrations  over  a  30-day  period  in  support  of 
this  decision.  Government  test  officials  believe  this 
pace  is  unrealistic  based  on  historical  data. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Common  Name:  MGUE  Increment  1 


Military  GPS  User  Equipment  (MGUE)  Increment  1 


The  Air  Force's  MGUE  program  plans  to  develop 
GPS  receivers  compatible  with  the  military's  next- 
generation  GPS  signal,  Military-Code.  The 
modernized  receivers  are  to  provide  U.S.  forces 
with  enhanced  position,  navigation,  and  time 
capabilities,  while  improving  resistance  to  existing 
and  emerging  threats,  such  as  jamming.  The 
program  consists  of  two  increments.  Increment  1, 
assessed  here,  leverages  technologies  from  the 
Modernized  User  Equipment  program  to  develop 
receivers  for  aviation,  maritime,  and  ground 
platforms. 


Source:  U.S.  Air  Force. 


Concept 


System  development  Production 


» 


▲ 

▲ 

▲ 

▲ 

▲ 

Program 

Preliminary 

GAO 

Development 

Production 

start 

design  review 

review 

start 

decision 

(4/12) 

(9/14) 

(1/16) 

(9/16) 

(TBD) 

Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  L-3,  Raytheon, 

As  of 

Latest 

Percent 

Rockwell  Collins  International 

08/2015 

change 

Program  Office:  El  Segundo,  CA 

Research  and  development  cost 
Procurement  cost 

N/A 

N/A 

$1,640.9 

$0.0 

N/A 

N/A 

Funding  needed  to  complete: 

Total  program  cost 

N/A 

$1,640.9 

N/A 

R&D:  $398.0  million 

Program  unit  cost 

N/A 

N/A 

N/A 

Procurement:  $0.0  million 

Total  quantities 

N/A 

N/A 

N/A 

Total  funding:  $398.0  million 

Acquisition  cycle  time  (months) 

N/A 

N/A 

N/A 

Procurement  quantity:  0 

The  latest  cost  data  reflect  the  funding  provided  for  MGUE  Increment  1  since  program  start  and 

exclude  expenditures  made  under  the  prior  Modernized  User  Equipment  Program. 

MGUE  is  scheduled  to  enter  development  in 
September  2016  with  three  of  its  five  critical 
technologies  mature.  In  order  to  begin  procuring 
MGUE  receivers  in  fiscal  year  201 8,  the  Air  Force 
eliminated  the  critical  design  review,  stating  that 
many  design  and  risk  assessments  were 
completed  earlier  during  design  and  test 
activities.  DOD  stated  that  conducting  a  design 
review,  as  recommended  by  GAO  in  September 
2015,  would  delay  the  program  by  6  to  12 
months.  However,  the  Army  stated  that  the 
current  prototypes  do  not  meet  all  Army 
requirements,  and  DOD's  test  authority  noted  that 
the  Air  Force  had  overstated  maturity.  Integration 
testing-during  which  problems  are  likely  to  be 
found-will  not  be  completed  until  2019. 
Consequently,  the  services  are  unlikely  to  have 
sufficient  knowledge  to  begin  procuring  MGUE 
receivers  in  2018. 


Attainment  of  Product  Knowledge 


Projected  as  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  ^ 

environment  ^ 

•  Demonstrate  all  critical  technologies  in  an  operational  r 

environment 

•  Complete  preliminary  design  review  £ 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

^  Knowledge  attained  ■*■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  MGUE  Increment  1 


MGUE  Increment  1  Program 

Technology  and  Design  Maturity 

The  MGUE  Increment  1  program  plans  to  enter 
system  development  with  three  of  its  five  critical 
technologies-military-code  acquisition  engine, 
military-code  cryptography,  and  selective  availability 
anti-spoofing  module  functionality-mature  and  the 
remainder-anti-spoofing  and  anti-tamper-nearing 
maturity.  These  maturity  levels  were  assessed 
during  a  November  2014  independent  technology 
readiness  assessment.  The  Director,  Operational 
Test  and  Evaluation  expressed  concerns  about  one 
of  the  Air  Force's  test  demonstrations  and 
emphasized  in  a  memorandum  to  the 
Undersecretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  that  the  Air  Force  had 
overstated  MGUE  maturity,  and  that  the 
demonstration  had  achieved  more  mixed  results 
than  the  service  indicated. 

The  program's  acquisition  strategy  may  limit  insight 
into  MGUE  design.  The  latest  approved  strategy 
eliminated  the  critical  design  review  as  the  Air  Force 
states  that  MGUE  design  has  been  adequately 
demonstrated.  Specifically,  the  Air  Force  stated  that 
this  review  was  unnecessary  as,  among  other 
things,  detailed  design  work  and  key  development 
and  design  risks  normally  reviewed  at  critical  design 
review  were  completed  for  MGUE's  preliminary 
design  review  in  September  2014.  In  addition,  the 
Assistant  Secretary  of  Defense  for  Acquisition 
stated  in  comments  to  GAO's  September  2015 
report  on  MGUE  that  conducting  a  critical  design 
review  would  require  MGUE  contractors  to  halt 
development  efforts  and  delay  the  program  by  6  to 
12  months,  delaying  the  expedited  fielding  of  MGUE 
receivers. 

The  program,  however,  carries  design-related  risks 
that  would  typically  be  addressed  in  a  critical  design 
review.  For  example,  in  an  April  2015  assessment 
of  the  MGUE  preliminary  design  review,  the  Office 
of  the  Undersecretary  of  Defense  for  Systems 
Engineering  noted  that  the  MGUE  preliminary 
designs  may  not  be  rigorous  enough  to  implement 
those  designs  beyond  the  initial  test  platforms  from 
the  military  services.  In  addition,  the  Army  has 
indicated  there  are  power  and  thermal 
incompatibility  issues  between  the  MGUE  ground 
receivers  and  Army  platforms.  Army  and  Navy 


officials  noted  that  their  respective  services  will 
need  to  conduct  further  work  to  develop  MGUE  to  a 
production-ready  status. 

Production  Maturity 

The  military  services  are  unlikely  to  have  the 
knowledge  to  begin  MGUE  procurement  at  the  start 
of  fiscal  year  2018.  The  National  Defense 
Authorization  Act  for  fiscal  year  2011  generally 
prohibits  DOD  from  obligating  or  expending  funds 
for  GPS  user  equipment  after  fiscal  year  201 7 
unless  that  equipment  is  capable  of  receiving 
military-code.  The  Secretary  of  Defense  may  waive 
this  limitation  under  certain  circumstances,  or 
certain  exceptions  may  apply.  Program  officials 
stated  that  the  Undersecretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  intends  to 
delegate  the  MGUE  production  decision  to  the 
military  services,  and  that  the  Air  Force  will  continue 
to  remain  responsible  for  receiver  certifications  and 
software  upgrades.  MGUE  integration  testing  with 
the  services'  lead  platforms  is  scheduled  to  continue 
until  late  2019.  This  phase  of  testing  commonly 
reveals  problems.  In  July  2015,  the  Army,  Navy,  and 
Marine  Corps  stated  that,  given  the  development 
and  testing  required  to  bring  MGUE  receivers  to 
production-ready  status,  they  cannot  currently 
determine  when  MGUE  will  be  ready  for 
procurement. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  noted  that  the  2014  technology 
readiness  assessment  rated  all  five  critical 
technologies  as  nearing  maturity  or  higher.  The 
program  stated  that  MGUE  will  conduct  full 
development  testing,  including  lab  and  platform 
testing,  on  initial  and  final  test  articles  for  early  risk 
reduction  and  to  verify  the  functional  baseline.  The 
program  also  noted  that,  in  parallel,  the  government 
will  assess  design  security  and  compatibility,  and 
the  MGUE  program  will  demonstrate  manufacturing 
readiness  on  a  pilot  line  and  provide  production 
representative  articles  for  operational  user 
evaluation.  The  program  office  also  provided 
technical  comments,  which  were  incorporated  as 
appropriate. 
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Lead  Component:  Air  Force 


Common  Name:  GPS  OCX 


Next  Generation  Operational  Control  System  (GPS  OCX) 


The  Air  Force's  GPS  OCX  is  to  replace  the  current 
GPS  satellite  ground  control  system.  GPS  OCX  will 
ensure  reliable  and  secure  delivery  of  position  and 
timing  information  to  military  and  civilian  users.  The 
Air  Force  plans  to  develop  GPS  OCX  in  blocks,  with 
each  block  delivering  upgrades  as  they  become 
available.  We  assessed  the  initial  three  blocks- 
Block  0  for  the  checkout  of  new  GPS  satellites; 
Block  1  for  satellite  control  and  basic  military 
signals;  and  Block  2  for  fully  modernized  military 
signals  and  additional  navigation  signals. 


Source:  U.S.  Air  Force. 


y  Concept 


System  development 


Production 


▲  A  A  A  A  A  A  A 
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(2/07)  (11/12)  (1/16)  (9/17)  (7/18)  (10/18)  (TBD)  (TBD) 


Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Raytheon 

As  of 

Latest 

Percent 

Program  office:  El  Segundo,  CA 

11/2012 

07/2015 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$3,552.5 

$3,737.0 

5.2% 

R&D:  $901 .3  million 

Procurement  cost 

$0.0 

$0.0 

0.0% 

Procurement:  $0.0  million 

Total  program  cost 

$3,552.5 

$3,737.0 

5.2% 

Total  funding:  $901.3  million 

Program  unit  cost 

$3,552,537 

$3,736,999 

5.2% 

Procurement  quantity:  0 

Total  quantities 

1 

1 

0.0% 

Acquisition  cycle  time  (months) 

55 

N/A 

N/A 

GPS  OCX  entered  system  development  in 
November  2012,  2  years  after  its  system 
development  contract  award,  with  all  critical 
technologies  nearing  full  maturity  and  after 
completing  a  preliminary  design  review.  The 
program  is  primarily  a  software  development 
effort,  and  according  to  program  officials,  has 
experienced  significant  development  difficulties 
due  to  poor  understanding  of  contract 
requirements,  resulting  in  $1.1  billion  in  cost 
growth  and  a  4  year  schedule  delay  so  far.  One 
effect  of  schedule  delays  is  that  future  GPS  III 
satellites  cannot  achieve  operational  capability 
until  OCX  Block  1  is  delivered.  The  Air  Force 
paused  development  in  2013  to  identify  and 
address  problems  in  the  program,  but  cost  and 
schedule  growth  persist  and  are  expected  to 
continue.  Unrealistic  cost  and  schedule  estimates 
limit  visibility  into  and  oversight  of  OCX  system 
development. 


Attainment  of  Product  Knowledge 
As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  ^ 

environment  ^ 

•  Demonstrate  all  critical  technologies  in  an  operational  q 

environment 

•  Complete  preliminary  design  review  0 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

0  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  GPS  OCX 


GPS  OCX  Program 

Technology  Maturity 

GPS  OCX  entered  system  development  in 
November  2012  with  its  14  critical  technologies 
nearing  maturity  and  after  an  August  2011 
preliminary  design  review.  However,  development 
work  began  more  than  2  years  prior  when  the 
program  awarded  the  system  development  contract 
in  February  2010.  According  to  program  officials, 
key  requirements-especially  for  cybersecurity-were 
poorly  understood  by  the  Air  Force  and  the 
contractor,  which  led  to  difficulty  in  developing 
capabilities  and  more  complexity  than  initially 
planned.  In  particular,  the  OCX  contractor  states  it 
did  not  fully  understand  the  cybersecurity 
implementation  requirements  that  help  ensure  DOD 
systems  can  resist  and  continue  to  operate  during 
cyber-attacks.  According  to  program  officials,  these 
requirements  issues  were  mostly  resolved  in  early 
2015. 

Design  and  Production  Maturity 

The  OCX  program  does  not  track  the  metrics  we 
use  to  measure  design  maturity,  such  as  the 
number  of  releasable  design  drawings,  as  it  is 
primarily  a  software  development  effort. 

The  OCX  contractor  failed  to  establish  consistent 
software  development  environments,  such  as  the 
computer  infrastructure  including  hardware, 
operating  systems,  and  databases,  across  the 
program  causing  high  software  defect  rates.  The 
program  has  not  yet  invested  the  time  and 
resources  to  conduct  assessments  and  determine  if 
any  systemic  issues  are  causing  these  defect  rates. 
We  recently  reported  that  the  high  defect  rate  may 
be  attributable  to  the  contractor's  undisciplined 
processes,  as  noted  by  the  OCX  program  office, 
independent  review  teams,  and  the  Defense 
Contract  Management  Agency  (DCMA). 

Other  Program  Issues 

OCX  development  contract  costs  have  more  than 
doubled-increasing  by  approximately  $1 .1  billion  to 
$1.98  billion-and  the  program's  schedule  has  been 
extended  by  5  years  from  estimates  established  at 
contract  award.  The  Air  Force  paused  development 
in  2013  to  identify  and  address  problems  in  the 
program,  but  cost  and  schedule  growth  have 
persisted.  Unless  problems  with  software 
development  are  addressed,  further  cost  and 


schedule  growth  is  likely.  According  to  DCMA 
officials,  the  program  has  a  history  of  overly 
optimistic  schedules.  Further,  the  program  assumed 
efficiencies  which  have  not  materialized  due  to  staff 
turnover,  process  changes,  and  poor  retention  of 
lessons  learned  from  past  software  development 
efforts.  As  a  result,  DOD  may  not  have  adequate 
insight  into  OCX  system  development. 

New  GPS  III  satellites  cannot  be  integrated  into  the 
constellation  or  achieve  operational  capability  until 
OCX  Block  1  is  delivered.  We  reported  in 
September  2015  that  an  independent  review  of  the 
program  found  Block  1  will  likely  be  delivered  in 
November  2020,  which  represents  an  additional 
year  delay  to  the  last  official  estimate.  However, 
recent  information  suggests  Block  1  delivery  may  be 
delayed  by  at  least  another  year  beyond  that.  As  a 
result,  it  is  likely  that  several  GPS  III  satellites  will  be 
launched  before  they  can  be  fully  tested  with  OCX 
as  needed.  The  Air  Force  plans  to  implement  a  GPS 
III  contingency  operations  capability,  which  will  allow 
for  limited  operational  capability  until  OCX  Block  1  is 
delivered. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  DOD 
stated  that  the  Air  Force  has  realized  a  3  year 
schedule  delay  since  program  inception  and  is 
currently  re-baselining  the  OCX  program,  extending 
the  schedule  by  an  additional  24  months.  This 
extension  is  based  on  poor  performance  by  the 
contractor,  and  in  accordance  with  guidance  from 
the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics.  Additionally,  a  new 
service  cost  position  is  currently  under 
development.  Technical  comments  were  also 
provided  by  the  program  office,  which  were 
incorporated  where  deemed  appropriate. 
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Lead  Component:  Air  Force 


Common  Name:  SDB  II 


Small  Diameter  Bomb  Increment  II  (SDB  II) 


The  Air  Force's  Small  Diameter  Bomb  Increment  II 
(SDB  II)  is  designed  to  provide  attack  capability 
against  mobile  targets  in  adverse  weather  from 
standoff  range.  It  combines  radar,  infrared,  and 
semiactive  laser  sensors  in  a  tri-mode  seeker  to 
acquire,  track,  and  engage  targets.  It  uses  airborne 
and  ground  data  links  to  update  target  locations,  as 
well  as  GPS  and  an  inertial  navigation  system  to 
ensure  accuracy.  SDB  II  will  be  integrated  with 
F-15E,  F/A-18E/F,  and  F-35  aircraft,  among  others. 


Source:  ©  2009  Raytheon  Company. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Raytheon  Missile 
Systems 

Program  office:  Eglin  AFB,  FL 

Research  and  development  cost 

As  of 
10/2010 

$1,764.6 

Latest 

09/2015 

$1,699.9 

Percent 

change 

-3.7% 

Funding  needed  to  complete: 

Procurement  cost 

$3,280.0 

$2,407.8 

-26.6% 

R&D:  $488.8  million 

Total  program  cost 

$5,044.6 

$4,107.7 

-18.6% 

Procurement:  $2,178.1  million 

Program  unit  cost 

$0,294 

$0,239 

-18.6% 

Total  funding:  $2,666.9  million 

Total  quantities 

17,163 

17,163 

0.0% 

Procurement  quantity:  16,856 

Acquisition  cycle  time  (months) 

72 

96 

33.3% 

SDB  II  entered  production  in  June  2015  with  all  of 
its  critical  technologies  mature  and  production 
processes  that  were  in  control,  but  with  an 
unstable  design.  The  program  still  has  significant 
testing  to  complete,  and  any  problems  it 
discovers  could  result  in  further  design  changes. 
Since  June  2015,  SDB  II  has  completed  three 
successful  flight  tests,  but  one  live  fire  flight  test 
failed.  The  program  was  not  able  to  identify  a  root 
cause  for  this  failure  and  plans  to  resume  flight 
tests  in  January  2016.  The  program  successfully 
completed  the  first  of  two  environmental 
qualification  tests.  The  second  test  is  scheduled 
for  the  second  quarter  of  fiscal  year  2016,  and 
officials  said  it  is  a  prerequisite  for  further 
production.  The  program  reported  an  eight  month 
delay  in  initial  operational  capability  due  to  design 
issues,  and  delays  with  the  F-35  program  have 
extended  low-rate  production. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 

•  Demonstrate  all  critical  technologies  in  an  operational 
environment 

•  Complete  preliminary  design  review 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 


•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  SDB  II 


SDB  II  Program 

Technology  and  Design  Maturity 

The  SDB  M's  four  critical  technologies-guidance 
and  control,  multi-mode  seeker,  net  ready  data  link, 
and  payload-are  mature.  They  reached  full  maturity 
by  the  program’s  low-rate  production  decision  in 
June  2015,  almost  5  years  after  development  start. 

SDB  II  experienced  design  changes  in  2015  and 
continues  to  experience  test  failures  that  could  lead 
to  further  design  changes.  All  of  the  program's 
design  drawings  have  been  released,  but  officials 
stated  that  the  system  will  require  design  changes 
and  drawings  updates  due  to  design  issues 
discovered  during  qualification  and  flight  testing. 
These  design  issues  were  responsible  for 
approximately  45  percent  of  the  program's  schedule 
delays. 

The  SBD  II  program  still  has  significant 
developmental  testing  to  complete,  and  any 
problems  it  discovers  could  result  in  further  design 
changes  and  costly  retrofits.  Following  the  low-rate 
production  decision  in  June  2015,  the  program 
successfully  completed  two  guided  flight  tests  and 
one  live  fire  flight  test.  A  second  live  fire  flight  test  hit 
its  intended  target,  but  the  warhead  did  not 
detonate.  The  program  was  not  able  to  identify  a 
root  cause  for  this  failure,  but  did  identify  potential 
failure  modes  that  could  cause  the  fuze  to  not  arm 
properly.  The  program  implemented  corrective 
actions  for  these  failure  modes  and  plans  to  conduct 
its  next  flight  test  in  January  2016.  The  program 
also  plans  to  conduct  additional  flight  tests  to  further 
refine  its  understanding  of  the  SBD  M's  three  attack 
modes.  This  developmental  testing  includes  10 
normal  attack  test  events,  six  laser  attack  events, 
and  six  coordinate  attack  events,  followed  by  a  28- 
shot  government  confidence  test  program  in  normal 
attack  mode. 

Production  Maturity 

In  June  2015,  SDB  II  program  was  approved  to 
enter  low-rate  production  after  demonstrating  its 
critical  processes  on  a  pilot  production  line,  testing  a 
production-representative  prototype,  and  reporting 
all  of  its  critical  manufacturing  processes  as  in 
control.  The  first  production  contract  was  for  144 
units.  Resolution  of  the  issues  discovered  during 
testing  may  require  manufacturing  changes  once 
the  design  changes  have  been  incorporated. 


Prior  to  SDB  Il's  low-rate  production  decision  it 
completed  the  first  part  of  its  corrosive  atmosphere 
environmental  test.  Officials  stated  that  this  test 
simulates  the  bomb's  exposure  to  various 
environmental  conditions  for  an  extended  period 
outside  of  its  protective  container.  This  successful 
test  followed  a  failed  test  in  June  2014.  The  second 
part  of  this  test  is  scheduled  to  be  completed  in  the 
second  quarter  of  fiscal  year  2016,  and  officials 
stated  that  it  is  a  prerequisite  for  the  Lot  2 
production  contract.  Officials  added  that  the 
configuration  of  the  first  two  production  lots  will  be 
slightly  different  due  to  this  environmental  test. 
Officials  stated  that  the  additional  environmental 
test  is  mainly  to  ensure  that  the  asset  will  be  safe  to 
operate  from  Navy  ships.  The  Navy  does  not  plan  to 
procure  any  units  until  Lot  4. 

Other  Program  Issues 

SDB  II  officials  stated  that  due  to  design  issues 
discovered  during  qualification  and  flight  testing,  the 
SDB  Il's  initial  operational  capability  date  has 
slipped  by  8  months  to  March  2018.  Delays  with  the 
F-35  program  have  also  caused  the  program  to 
extend  its  low-rate  production,  which  will  now 
include  around  59  percent  of  the  total  procurement. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment, 
program  officials  stated  that  software  development, 
technical  discoveries,  and  flight  test  failure 
investigation  activities  have  delayed  progress.  The 
program  office  is  actively  engaged  with  the 
manufacturer  regarding  a  disciplined  approach  to 
SDB  II  software  development,  reinvigorating  quality, 
and  concentrated  focus  on  the  work  required  to 
return  to  flight  testing.  SDB  II  remains  event  based 
and  will  execute  tests  when  ready. 

The  SDB  II  multi-mode  seeker  continues  to 
demonstrate  excellent  performance  in  guided  flight 
testing  and  captive  flight  tests.  The  seekers  ability  to 
correctly  classify  targets  (wheeled,  tracked,  boat)  in 
challenging  environments  and  weather  was 
demonstrated  on  the  guided  flight  tests  conducted 
during  2015.  To  date  the  seeker  has  demonstrated 
target  classification  capability  that  exceeds  95 
percent  accuracy  rate.  This  capability  is  unique  to 
SDB  II  and  provides  the  operator  greater  targeting 
flexibility  and  accuracy  against  moving  targets 
operating  in  adverse  environmental  conditions. 
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Lead  Component:  Air  Force 


Common  Name:  Space  Fence  Inc  1 


Space  Fence  Ground-Based  Radar  System  Increment  1 


The  Air  Force's  Space  Fence  Increment  1  program 
is  developing  a  large  ground-based  radar  intended 
to  detect  and  track  objects  in  low  and  medium  Earth 
orbit  and  provide  this  information  to  a  space 
surveillance  network.  Space  Fence  is  designed  to 
use  high  radio  frequencies  to  detect  and  track  more 
and  smaller  objects  than  previous  systems.  The  Air 
Force  awarded  a  contract  for  the  development  and 
production  of  the  first  radar  site  in  June  2014,  and 
included  a  second  radar  site  as  an  option  which,  if 
exercised,  will  be  a  separate  program. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Lockheed  Martin 

As  of 

Latest 

Percent 

MST 

04/2014 

10/2015 

change 

Program  office:  Hanscom  AFB,  MA 

Research  and  development  cost 

$1,614.3 

$1,580.0 

-2.1% 

Funding  needed  to  complete: 

Procurement  cost 

$0.0 

$0.0 

0.0% 

R&D:  $495.6  million 

Total  program  cost 

$1,614.3 

$1,580.0 

-2.1% 

Procurement:  $0.0  million 

Program  unit  cost 

$1,614,299 

$1,579,994 

-2.1% 

Total  funding:  $495.6  million 

Total  quantities 

1 

1 

0.0% 

Procurement  quantity:  0 

Acquisition  cycle  time  (months) 

124 

115 

-7.3% 

Since  our  last  assessment,  the  Space  Fence 
program  has  fully  matured  all  its  critical 
technologies  through  demonstrations  with  a 
prototype  array  or  modeling  and  simulation.  The 
program  completed  its  critical  design  review  in 
June  2015  and  has  completed  all  of  its  design 
drawings  but  has  yet  to  demonstrate  a  system- 
level  integrated  prototype.  The  program  plans  to 
verify  the  design  through  testing  on  a  prototype 
with  some  production-representative  elements 
beginning  in  2016.  To  accommodate  the  projected 
increase  in  the  volume  of  data  generated  by 
Space  Fence,  a  new  data  processing  system  is 
being  developed  by  the  Air  Force  in  a  separate 
acquisition  program.  The  Air  Force  will  achieve 
initial  capability  when  the  first  site  becomes 
operational,  but  full  capability  will  only  be 
achieved  once  the  second  site  is  operational. 


Attainment  of  Product  Knowledge 


As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  ^ 

environment  ^ 

•  Demonstrate  all  critical  technologies  in  an  operational  £ 

environment 

•  Complete  preliminary  design  review  £ 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings  0 

•  Test  a  system-level  integrated  prototype  o 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  Space  Fence  Inc  1 


Space  Fence  Inc  1  Program 

Technology  Maturity 

A  technology  readiness  assessment  completed  in 
February  2015  showed  that  all  seven  of  the 
program's  critical  technologies  are  fully  mature.  This 
maturity  was  achieved  through  integration  of  a 
prototype  array  or  through  modeling  and  simulation. 
Most  of  the  technologies  provide  the  capabilities  for 
transmitting  and  receiving  radar  signals  from  the 
large  array  planned  for  Space  Fence.  According  to 
program  officials,  this  array  will  be  one  of  the  largest 
phased  array  radars  ever  built  and  will  be 
significantly  larger  in  size,  and  generate  significantly 
more  data,  than  the  system  it  replaces.  An  array  this 
size  presents  multiple  challenges,  such  as 
calibrating  and  managing  multiple  radar  beams  and 
processing  the  significantly  increased  volume  of 
tracking  data. 

Design  and  Production  Maturity 

Space  Fence  completed  its  critical  design  review  in 
June  2015  with  100  percent  of  its  design  drawings 
complete.  The  final  radar  design  will  be  15  percent 
smaller  than  the  contractor  had  initially  planned. 
Testing  of  the  initial  design  showed  better 
performance  than  expected,  and  the  contractor 
found  that  it  could  achieve  the  Air  Force's  required 
performance  with  a  smaller  radar.  The  Air  Force 
confirmed  these  findings  by  independently 
examining  the  test  results.  The  program  plans  to 
verify  the  design  through  testing  on  a  prototype  with 
some  production-representative  elements  beginning 
in  2016.  This  prototype  will  be  approximately  10 
percent  of  the  size  of  the  final  radar,  but  will 
demonstrate  up  to  70  percent  of  the  system 
requirements.  According  to  the  program,  the 
testbed  prototype  is  not  a  contract  requirement;  the 
contractor  elected  to  build  it  as  part  of  the 
development  process. 

Award  of  the  development  and  production  contract 
marked  the  start  of  production.  Space  Fence  is 
tracking  the  maturity  of  two  manufacturing 
processes  for  components  of  the  transmit  and 
receive  modules  critical  to  the  radar's  production. 
These  processes  have  not  been  demonstrated  as  in 
control,  as  recommended  by  best  practices,  but  are 
at  the  level  of  maturity  required  in  the  system 
development  contract. 


Other  Program  Issues 

The  first  Space  Fence  site,  or  increment,  is 
expected  to  meet  the  Air  Force's  requirements  for 
initial  operational  capability,  but  full  capability  will 
only  be  achieved  once  a  second  site  is  operational. 
Development  and  production  of  the  second  radar 
site,  which  would  be  Space  Fence  Increment  2,  is  a 
contract  option.  According  to  the  program  office,  the 
Air  Force  is  currently  analyzing  space  threats  to 
determine  how  a  potential  second  site  would 
contribute  to  mitigating  these  threats.  According  to 
the  Air  Force,  the  option  for  the  second  site  has  to 
be  exercised  before  the  negotiated  fixed  price 
option  for  the  second  increment  expires  in  August 
2018. 

The  Joint  Space  Operations  Center  (JSpOC)  at 
Vandenberg  AFB  is  acquiring  a  new  data 
processing  capability-JSpOC  Mission  System 
(JMS)-designed  to  enable  processing  of  the 
increased  volume  of  data  expected  from  Space 
Fence.  Currently,  JMS  is  scheduled  to  become 
operational  by  December  2016,  enabling  input  and 
processing  of  data  from  Space  Fence  to  support  its 
operational  testing  and  initial  capability.  According 
to  the  Space  Fence  program  office,  the  Space 
Fence  and  JMS  programs  have  developed  an 
interface  control  document  to  ensure  compatibility 
and  have  planned  joint  test  activities  for  the  two 
systems. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  noted  that  manufacturing  has  begun 
for  the  production  of  line  replaceable  units  for  the 
integration  test  bed  prototype.  The  prototype  is 
undergoing  installation  and  check-out  of  the  radar 
transmit  and  receive  hardware  and  software.  The 
prototype  should  track  its  first  resident  space  object 
in  January  and  be  fully  operational  in  March  2016. 
In  addition,  the  program  office  noted  that  significant 
construction  progress  has  been  made  at  the  Space 
Fence  Increment  1  site.  The  program  also  has 
ongoing  integration  risk  reduction  efforts  between 
Space  Fence  and  JMS.  The  contractor  has 
connected  to  the  JMS  interface  development 
environment  and  is  performing  weekly  risk  reduction 
testing.  The  second  software  build  is  complete  and 
the  third  build  is  underway. 
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Lead  Component:  Air  Force 


Common  Name:  3DELRR 


Three-Dimensional  Expeditionary  Long-Range  Radar  (3DELRR) 

The  Air  Force's  3DELRR  is  being  developed  as  a 
long-range,  ground-based  sensor  for  detecting, 
identifying,  tracking,  and  reporting  aerial  targets, 
including  highly  maneuverable  and  low  observable 
targets.  The  system  intends  to  provide  real-time 
data  and  support  a  range  of  operations  in  all  types 
of  weather  and  terrain.  It  will  replace  the  Air  Force's 
AN/TPS-75  radar  systems,  which  is  experiencing 
maintainability  and  sustainability  issues. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  TBD 

As  of 

Latest 

Percent 

Program  office:  Hanscom  Air  Force 

07/2015 

change 

Base,  MA 

Research  and  development  cost 

N/A 

$398.9 

N/A 

Funding  needed  FY  2016  to  FY  2020 

Procurement  cost 

N/A 

$983.0 

N/A 

R&D:  $178.7  million 

Total  program  cost 

N/A 

$1,381.9 

N/A 

Procurement:  $373.3  million 

Program  unit  cost 

N/A 

$39,483 

N/A 

Total  funding:  $552.1  million 

Total  quantities 

N/A 

35 

N/A 

Procurement  quantity:  11 

Acquisition  cycle  time  (months) 

N/A 

151 

N/A 

3DELRR  entered  system  development  in  October 
2014  with  all  of  its  critical  technologies  nearing  full 
maturity  and  awarded  a  system  development 
contract  the  following  month.  However,  bid 
protests  delayed  the  start  of  development  work, 
and,  as  a  result  of  issues  raised  by  these  protests, 
the  Air  Force  re-opened  the  competition  and 
plans  to  award  a  new  system  development 
contract  in  the  second  quarter  of  fiscal  year  201 6. 
Prior  to  system  development  start,  the  program 
took  steps  to  reduce  technical  risk  and  program 
costs  by  conducting  system-level  competitive 
prototyping  and  analyzing  the  tradeoffs  between 
costs  and  requirements.  The  3DELRR  program’s 
remaining  development  risks  may  vary  to  some 
extent  based  on  the  contractor  design  selected 
for  system  development,  although  software 
integration  will  be  a  risk  regardless  of  the 
contractor. 
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Attainment  of  Product  Knowledge 

As  of  January  2016 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  ^ 

environment  ^ 

•  Demonstrate  all  critical  technologies  in  an  operational  r 

environment 

•  Complete  preliminary  design  review  £ 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

^  Knowledge  attained  ■*■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  3DELRR 


3DELRR  Program 

Technology  and  Design  Maturity 

The  3DELRR  program  entered  system  development 
in  October  2014  with  all  of  its  critical  technologies 
approaching  full  maturity.  The  program  plans  to  fully 
mature  its  critical  technologies  through  the  system 
development  phase  by  demonstrating  them  in  a 
realistic  environment.  According  to  program 
officials,  the  critical  technologies  will  vary  based  on 
the  contractor  selected  for  system  development. 

While  the  3DELRR  program's  specific  development 
risks  may  vary  to  some  extent  based  on  the  design 
of  the  contractor  selected  for  system  development, 
the  program  is  tracking  a  number  of  overall 
development  risks.  For  example,  the  3DELRR’s 
planned  design  is  software-intensive,  and  software 
development  was  identified  by  program  officials  as 
a  risk  because,  if  not  performed  adequately, 
subsequent  integration  of  hardware,  software,  and 
firmware  could  be  delayed,  resulting  in  additional 
cost.  Program  officials  also  stated  that  integrating 
the  extensive  amount  of  re-used  software  code 
contributed  to  the  level  of  risk,  but  noted  that  each 
contractor  is  planning  to  mature  and  test  software 
prior  to  installation  and  integration  into  the  system. 
In  addition,  3DELRR  is  expected  to  use  a  new 
semiconductor  technology,  which  could  pose  cost  or 
schedule  risks.  Specifically,  the  system  is  expected 
to  use  gallium  nitride-based  modules  for  the 
individual  radiating  elements  key  to  transmitting  and 
receiving  electromagnetic  signals,  rather  than  the 
legacy  gallium  arsenide  transmit/receive  modules. 
While  the  use  of  gallium  nitride  may  present  some 
risks  for  the  program,  as  long-term  reliability  and 
performance  of  this  material  are  unknown  and  could 
affect  radar  sensitivity  and  power  requirements,  it 
has  the  potential  to  provide  higher  efficiency  with 
lower  power  and  cooling  demands  than  legacy 
semiconductor  technology. 

Prior  to  the  start  of  system  development,  the 
3DELRR  program  conducted  a  number  of  risk 
reduction  efforts  in  the  technology  development 
phase.  For  example,  the  program  conducted 
system-level  competitive  prototyping,  held 
preliminary  design  reviews  with  multiple  contractors, 
and  conducted  capability  demonstrations. 
According  to  program  officials,  these  risk  reduction 


efforts  allowed  the  program  to  mature  critical 
technologies,  refine  technical  requirements  and  cost 
estimates,  and  assess  manufacturing  readiness. 

Other  Program  Issues 

Performance  of  the  system  development  contract 
awarded  to  Raytheon  in  October  2014  was 
suspended  as  a  result  of  bid  protests  by  Lockheed 
Martin  and  Northrop  Grumman.  In  January  2015, 
GAO  dismissed  the  protests  when  the  agency 
agreed  to  take  corrective  actions  to  address  the 
issues  raised  in  these  protests.  Program  officials 
stated  that  the  program  re-entered  source  selection 
in  May  2015,  and  they  plan  to  award  a  new  system 
development  contract  in  the  second  quarter  of  fiscal 
year  2016. 

Program  Office  Comments 

The  program  office  concurred  with  this  assessment 
and  provided  technical  comments,  which  were 
incorporated  where  appropriate. 
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Lead  Component:  Air  Force 


Common  Name:  APT 


Advanced  Pilot  Training  (APT) 


The  Air  Force's  Advanced  Pilot  Training  (APT) 
program  is  being  developed  to  replace  the  T-38C 
aircraft  and  associated  ground  based  training 
system  currently  used  to  meet  the  Air  Force's 
advanced  fighter  pilot  training  needs.  The  APT  will 
close  training  gaps  which  the  T-38  cannot  fully 
address  with  the  introduction  of  the  4th  and  5th 
generation  fighter  aircraft. 


Source:  U.S.  Air  Force. 

Current  Status 

In  October  2009,  the  Air  Force  identified  a  gap  in  its  aircraft  training  beginning  in  2018.  In  May  2010,  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  approved  the  Air  Force's  plans  to 
conduct  an  analysis  of  alternatives.  In  June  2011,  the  analysis  of  alternatives  was  recommended  that  the 
existing  training  aircraft,  the  T-38,  be  replaced  because  a  modification  program  would  not  be  cost  effective, 
address  all  the  identified  capability  gaps,  and  would  leave  the  USAF  with  aging  airframes.  The  Air  Force 
plans  to  release  a  request  for  proposals  in  the  fourth  quarter  of  fiscal  year  2016  and  award  a  fixed-price 
development/production  contract  in  2017.  According  to  the  Air  Force,  all  required  technology  elements  have 
mature  solutions.  In  October  2015,  the  Joint  Requirements  Oversight  Council  approved  the  Capability 
Development  Document  for  the  APT.  The  Air  Force  has  requested  waivers  for  some  of  the  mandatory 
certifications  required  prior  to  the  start  of  system  development,  as  the  program  does  not  plan  to  conduct  a 
preliminary  design  review  or  pursue  competitive  prototyping  before  milestone  B.  According  to  program 
officials,  all  technical  risks  are  low  and  all  systems  will  have  completed  substantial  flight  tests  prior  to 
proposal  submittals.  Each  offeror  will  be  expected  to  present,  as  part  of  their  proposal,  actual  flight  test  data 
that  their  aircraft  performance  parameters  are  being  met.  In  our  previous  work,  we  have  found  that 
acquisition  programs  that  successfully  complete  a  preliminary  design  review  prior  to  starting  system 
development  typically  have  better  costs  and  schedule  outcomes.  Initial  operational  capability  for  the  APT  is 
currently  expected  in  the  fourth  quarter  of  fiscal  year  2024.  The  APT  program  does  not  have  an  approved 
acquisition  program  budget  (APB)  at  this  time;  therefore,  the  estimated  development  cost  shown  below  was 
obtained  from  the  fiscal  year  2016  President's  Budget  Request. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2016  dollars): 

Total  program:  TBD 

Research  and  development  (fiscal  years  2015-2020):  $600.4  million 
Procurement:  TBD 
Quantity:  TBD 

Next  Major  Program  Event:  Development  request  for  proposal  release  decision,  fourth  quarter  fiscal  year 
2016 

Program  Office  Comments:  According  to  the  program  office,  the  proposed  acquisition  strategy  was 
approved  at  an  Acquisition  Strategy  Panel  in  July  2015.  The  program  continues  to  gather  pre-solicitation 
information  from  industry  to  refine  the  request  for  proposals  (RFP).  The  APT  program  expects  to  release  an 
RFP  in  late  fiscal  year  2016  with  contract  award  and  milestone  B  expected  in  late  fiscal  year  2017. 
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Lead  Component:  Air  Force 


Common  Name:  F-15  EPAWSS 


F-15  Eagle  Passive  /  Active  Warning  and  Survivability  System  (F-15  EPAWSS) 

The  Air  Force's  F-15  EPAWSS  program  plans  to 
upgrade  the  electronic  warfare  system  on  fielded  F- 
15  aircraft.  The  Air  Force  will  use  an  incremental 
approach  to  deliver  survivability  improvements  that 
include  enhancing  the  F-15's  ability  to  detect, 
identify,  locate,  deny,  degrade,  disrupt,  and  defeat 
air  and  ground  threat  systems.  These  improvements 
will  enable  operations  in  current  and  future  threat 
environments  and  allow  the  F-15  fleet  to  remain 
viable  into  the  2040  timeframe.  We  assessed  the 
first  of  two  planned  increments. 


Current  Status 

In  August  2015,  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  approved  the 
program's  entry  into  technology  development  with  system  development  scheduled  to  begin  in  August  201 6. 
The  program's  request  to  waive  competitive  prototyping  requirements  prior  to  system  development  was 
approved  in  February  2015.  A  preliminary  design  review  is  currently  planned  for  July  2016.  The  Air  Force  is 
taking  an  incremental  acquisition  approach  to  EPAWSS.  The  first  increment  replaces  the  F-15's  existing, 
internal  electronic  warfare  system.  The  second  increment,  contingent  upon  funding  availability,  adds  a  new 
towed  decoy  and  associated  countermeasures.  The  Air  Force  awarded  a  technology  development  contract 
to  Boeing  in  September  2015.  Under  this  contract,  Boeing  will  have  primary  responsibility  for  system 
integration  and  has  subcontract  the  development  of  the  electronic  warfare  subsystem  to  a  vendor  that 
Boeing  competitively  selected  in  May  2015. 

The  Air  Force  plans  to  leverage  non-developmental  electronic  warfare  technologies  and  components 
currently  used  in  other  military  aircraft  to  create  a  new  electronic  warfare  system  capable  of  protecting  the 
F-15  against  advanced  enemy  threats.  However,  the  Air  Force  identified  the  digital  receiver  technology-both 
hardware  and  associated  software-as  a  critical  technology  that  the  program  does  not  expect  to  fully  mature 
prior  to  the  start  of  system  development.  According  to  program  officials,  software  development  and 
integration  of  the  electronic  warfare  system  with  all  other  on-board  and  off-board  systems  are  risk  areas  that 
could  affect  the  Air  Force's  ability  to  have  the  required  number  of  assets  ready  in  time  for  the  projected  initial 
operational  capability  in  2021. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2016  dollars): 

Total  program:  $4,212.1  million 

Research  and  development:  $828.0  million 

Procurement:  $3,384.1  million 

Quantity:  10  (development),  403  (procurement) 

Next  Major  Program  Event:  Preliminary  design  review,  July  2016 

Program  Office  Comments:  According  to  Air  Force  officials,  the  program  is  implementing  a  strategy  to 
leverage  mature  non-developmental  components  as  a  key  element  of  the  program's  efforts  to  drive  positive 
schedule,  affordability  and  risk  outcomes.  The  program  plans  to  demonstrate  digital  receiver  critical 
technology  prior  to  entry  into  system  development. 
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Lead  Component:  Air  Force 


Common  Name:  JSTARS  Recap 


Joint  Surveillance  Target  Attack  Radar  System  Recapitalization 

(JSTARS  Recap) 

The  JSTARS  aircraft  is  a  manned  airborne  Battle 
Management  Command  and  Control  (BMC2)  system 
providing  near  real-time  surveillance  and  targeting 
information  on  moving  and  stationary  ground  targets. 

The  Air  Force's  JSTARS  Recap  program  is  an  effort 
to  replace  aging,  legacy  JSTARS  aircraft  initially 
fielded  in  the  early  1990s.  The  JSTARS  Recap 
acquisition  program  seeks  to  greatly  reduce  aircraft 
operating  and  sustainment  costs,  replace  and 
improve  JSTARS  functionality,  and  minimize 
development  and  integration  costs. 


Source:  U.S.  Air  Force. 


Current  Status 

The  JSTARS  Recap  program  completed  its  materiel  development  decision  in  May  2015  and  received 
following  a  seven-month  material  solution  analysis  phase,  the  program  was  granted  a  milestone  A  decision 
by  the  Under  Secretary  of  Defense,  Acquisition,  Technology,  and  Logistics  on  December  10,  2015.  The 
program  intends  to  use  an  in-production,  commercial-derivative  aircraft  integrated  with  a  sophisticated 
surface  search  radar,  BMC2,  and  broad-spectrum  communication  subsystems.  The  program  also  plans  to 
deliver  an  adaptable,  open  architecture  solution  to  reduce  future  upgrade  costs  and  increase  competition 
across  the  JSTARS  Recap  lifecycle.  In  August  2015,  three  firm-fixed  price  pre-engineering  and 
manufacturing  development  contracts  were  awarded  to  Lockheed  Martin,  Northrop  Grumman,  and  Boeing. 
According  to  the  program,  each  of  these  contracts  includes  a  provision  for  conducting  a  system 
requirements  review  as  well  as  options  for  conducting  system  functional  reviews,  system-level  preliminary 
design  reviews,  and  building  subsystem  prototype  demonstrators  of  the  BMC2,  radar,  and  communication 
subsystems.  These  subsystems  are  considered  high  risk  due  to  systems  integration  challenges.  According 
to  a  program  official,  JSTARS  Recap  exercised  the  options  in  December  2015  and  is  moving  toward 
preliminary  design  reviews  and  subsystem  prototype  demonstrations  in  the  spring/summer  of  2016.  The 
program  has  not  identified  any  immature  technologies  and  stated  that  all  major  subsystem  hardware,  and 
the  majority  of  the  BMC2  software,  currently  exist  and  are  fielded.  These  subsystems  have  not,  however, 
been  integrated  together  into  a  single  aircraft-based  system.  The  program  plans  to  conduct  system-level 
preliminary  design  reviews  with  each  contractor  prior  to  selecting  a  single  contractor  and  then  starting 
development,  a  change  from  last  year's  assessment.  Initial  operational  capability  for  JSTARS  Recap  is 
expected  in  the  fourth  quarter  of  fiscal  year  2023. 

Estimated  Total  Program  Cost  (fiscal  year  2016  dollars): 

Total  program:  $6,486.35  million 
Research  and  development:  $2,422.38  million 
Procurement:  $4,063.97  million 
Quantity:  17 

Next  Major  Program  Event:  Request  for  proposal  decision  review,  September  2016 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 
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Lead  Component:  Air  Force 


Common  Name:  PAR 


Presidential  Aircraft  Recapitalization  (PAR) 


The  PAR  program  plans  to  replace  the  current  VC- 
25A  fleet  with  a  new  fleet  of  aircraft  to  support  the 
President  of  the  United  States  as  Head  of  State, 
Chief  Executive,  and  Commander  in  Chief.  The  PAR 
aircraft  will  be  a  four  engine  wide-body,  commercial 
derivative  aircraft,  uniquely  modified  to  provide  the 
President,  staff,  and  guests  with  safe  and  reliable  air 
transportation  with  the  equivalent  level  of  security 
and  communications  capability  available  in  the 
White  House. 


Source:  U.S.  Air  Force. 


Current  Status 

The  PAR  program  acquisition  strategy  was  approved  in  September  2015  after  several  years  of  analysis 
related  to  risk  reduction,  requirements,  sustainment,  and  technology  and  manufacturing  maturity.  According 
to  officials,  the  milestone  decision  authority  approved  the  PAR  program  to  release  a  request  for  proposal  to 
Boeing  for  design  and  risk  reduction  studies.  The  contract  award  for  this  effort  is  planned  for  January  2016. 
The  program  will  seek  a  waiver  from  the  requirement  to  conduct  competitive  prototyping  before  entering 
system  development  and  to  award  a  sole-source  contract  to  Boeing  for  two  747-8  aircraft,  which  will  then  be 
modified  to  meet  required  capabilities  with  existing  technologies.  Program  officials  stated  that  they  will 
participate  in  Boeing  supplier  selections  for  the  subsystems  and  intend  to  acquire  data  rights  to  enable 
competition  to  the  maximum  extent  practicable  for  future  modifications  and  sustainment  activities.  Program 
officials  acknowledge  risks  associated  with  the  integration  of  these  technologies  but  stated  that  the  majority 
of  the  mission-related  systems  required  have  worked  together  before  on  different  platforms,  and  many  of 
these  systems  have  legacy  or  related  equivalents  on  the  current  VC-25A  fleet.  The  two  aircraft  will  be 
modified  and  tested  in  a  phased  approach.  Once  development  is  completed,  the  aircraft  will  be  delivered  as 
fully  capable  to  support  presidential  missions,  currently  planned  for  fiscal  year  2024. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2016  dollars): 

Total  program  (fiscal  years  2010-2020):  $3,210.6  million 
Research  and  development  (fiscal  years  2010-2020):  $1,987.1  million 
Procurement  (fiscal  years  2017-2020):  $282.2million 
Quantity:  2 

Next  Major  Program  Event:  Engineering  manufacturing  and  development  request  for  proposal  release 
decision,  July  2016 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  as  appropriate. 
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Lead  Component:  Air  Force 


Common  Name:  WSF 


Weather  System  Follow-on  (WSF) 

The  Air  Force's  Weather  System  Follow-on  (WSF)  is 
expected  to  be  DOD's  future  weather  satellite 
system.  The  WSF  program  is  to  be  comprised  of  a 
group  of  systems  to  provide  timely,  reliable  remote 
sensing  capabilities  that  will  make  global 
environmental  observations  of  atmospheric, 
terrestrial,  oceanographic,  and  solar-geophysical 
conditions,  and  meet  other  requirements  validated 
by  the  Joint  Requirements  Oversight  Council. 


Source:  U.S.  Air  Force. 


Current  Status 

Following  two  program  cancellations,  WSF  is  the  third  effort  to  provide  space-based  meteorological  and 
oceanographic  sensing  capabilities  to  replace  the  current  Defense  Meteorological  Satellite  Program.  As 
currently  envisioned,  WSF  is  to  satisfy  3  of  11  necessary  capability  requirements  as  determined  by  a  Space 
Based  Environmental  Monitoring  Analysis  of  Alternatives  completed  in  October  2013.  The  three  capability 
requirements  are  to  address  ocean  surface  vector  winds,  tropical  cyclone  intensity,  and  low  earth  orbit 
energetic  charged  particle  characterization,  and  are  to  be  completed  in  two  phases.  According  to  officials, 
Phase  1  is  to  be  a  technology  demonstration  and  is  expected  to  use  a  currently  available  microwave 
payload  to  partially  address  ocean  surface  vector  winds  and  tropical  cyclone  intensity.  The  Air  Force, 
through  its  Operationally  Responsive  Space  Office,  will  accelerate  the  acquisition  process  for  Phase  1.  The 
technology  demonstration  is  expected  to  launch  in  late  2017  or  early  2018.  Phase  2,  according  to  officials,  is 
the  WSF  objective  system. 

The  objective  system  will  use  microwave  sensor  technology  and  is  envisioned  as  a  single  polar-orbiting 
satellite.  DOD  officials  expect  to  launch  the  satellite  in  2022  and  to  acquire  a  replacement  satellite  every  5  to 
7  years.  The  program  office  expects  to  submit  the  acquisition  strategy  to  the  Office  of  the  Undersecretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics  in  January  2016.  According  to  program  officials,  DOD  is 
reviewing  the  remaining  capability  gaps  focusing  on  the  Joint  Requirements  Oversight  Council's  validated 
highest  priority  needs-cloud  characterization  and  theater  weather  imagery. 

The  program  plans  to  enter  development  with  five  of  the  six  critical  technologies-electronic  polarization  basis 
rotation,  digital  polarimetric  receiver,  digital  back-end,  internal  calibration,  and  radio  frequency  interference 
mitigation-nearing  maturity.  The  final  technology-the  bearing  and  power  transfer  assembly-is  expected  to 
be  mature  at  development  start. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2016  dollars): 

Total  program:  TBD 

Research  and  development  (fiscal  years  2012-2020):  $750.4  million 
Procurement:  TBD 
Quantity:  TBD 

Next  Major  Program  Event:  System  development  start,  TBD 

Program  Office  Comments:  The  WSF  program  officials  provided  technical  comments,  which  were 
incorporated  as  appropriate. 
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Joint  DOD-wide  Programs  Summary 


We  performed  in-depth  assessments  on  two  of  the  three  “joint”  or  DOD-wide  major  defense  acquisition 
programs  in  the  current  portfolio — the  F-35  Joint  Strike  Fighter  or  Lighting  II  program  and  Joint  Light  Tactical 
Vehicle.  These  two  programs  currently  estimate  a  need  of  more  than  $253  billion  in  funding  to  complete  their 
acquisition.  We  determined  cost  and  schedule  change  from  first  full  estimates  for  both  programs,  and  together 
they  have  experienced  more  than  $110  billion  in  cost  growth  and  average  schedule  delays  of  approximately  40 
months.  All  of  the  cost  growth  is  attributable  to  the  F-35,  and  over  70  percent  of  the  growth  occurred  more  than 
5  years  ago.  Of  these  two  programs,  the  F-35  has  completed  all  the  activities  associated  with  applicable 
knowledge-based  best  practices  we  assess  but  did  so  more  than  7  years  after  its  production  start. 


Acquisition  Phase  and  Size  of  the 
Two  Programs  Assessed 


Production 


System  development 
Technology  development 


•  Cost  growth  of  more  than  15  percent  and/or 
schedule  delays  of  more  than  6  months 

•  Cost  growth  of  15  percent  or  less  and  schedule 
delays  of  6  months  or  less 

No  first  full  estimate  available 


Note:  Bubble  size  is  based  on  each  program’s  currently  estimated 
future  funding  needed. 


Currently  Estimated  Acquisition  Cost 
for  the  Two  Programs  Assessed 


Summary  of  Knowledge  Attained  to  Date  for 
Programs  Beyond  System  Development  Start 


Fiscal  year  2016  dollars  in  billions 


JLTV 

(in  production)  Program 
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name 


F-35 


O 


Cost  and  Schedule  Growth  on  Two  Programs 
in  the  Current  Portfolio 


Knowledge 

Point  (KP)  1  Knowledge  Knowledge 

Resources  Point  2  Point  3 

and  Product  Manufacturing 

requirements  design  processes  are 

match  is  stable  mature 


JLTV 


O 


All  applicable  knowledge  practices  were 
completed 

One  or  more  applicable  knowledge  practices 
were  not  completed 

All  knowledge  practices  were  not  applicable 
Information  not  available  for  one  or  more 
knowledge  practice 


Fiscal  year  2016  dollars  in  billions 


$87.3 


$21.0 

$42.8 


Research  and  Procurement 
development  costs 
costs 


$208.6 


$110.5 


$253.5 


Total 

acquisition 

costs 


□  Growth  from  first 
full  estimate 


[  First  full 
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acquisition 
cycle  time 

(in  months) 


1 - 

40 

150 

Note:  In  addition  to  research  and  development  and  procurement 
costs,  total  acquisition  cost  includes  acquisition  related  operations 
and  maintenance  and  system-specific  military  construction  costs. 


Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 
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Lead  Component:  DOD-wide 


Common  Name:  F-35 


F-35  Lightning  II  Program  (F-35) 


DOD's  F-35  program  is  developing  a  family  of 
stealthy,  strike  fighter  aircraft  for  the  Navy,  Air 
Force,  Marine  Corps,  and  U.S.  allies,  with  the  goal 
of  maximizing  commonality  to  minimize  life-cycle 
costs.  The  carrier-suitable  variant  will  complement 
the  Navy  F/A-18E/F.  The  Air  Force  variant  is 
expected  to  replace  the  air-to-ground  attack 
capabilities  of  the  F-16  and  A-10,  and  complement 
the  F-22A.  The  short  take-off  and  vertical  landing 
(STOVL)  variant  is  expected  to  replace  the  Marine 
Corps’  F/A-18  and  AV-8B  aircraft. 


Source:  ©  Lockheed  Martin. 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  Lockheed 

As  of 

Latest 

Percent 

Martin,  Pratt  and  Whitney 

10/2001 

12/2014 

change 

Program  office:  Arlington,  VA 

Research  and  development  cost 

$41,817.7 

$62,882.7 

50.4% 

Funding  needed  to  complete: 

Procurement  cost 

$185,525.6 

$272,943.7 

47.1% 

R&D:  $2,915.6  million 

Total  program  cost 

$229,285.3 

$339,996.9 

48.3% 

Procurement:  $225,107.1  million 

Program  unit  cost 

$80,002 

$138,379 

73.0% 

Total  funding:  $230,649.6  million 

Total  quantities 

2,866 

2,457 

-14.3% 

Procurement  quantity:  2,226 

Acquisition  cycle  time  (months) 

175 

237 

35.4% 

All  of  the  program's  critical  technologies  are  now 
considered  fully  mature.  One  former  critical 
technology,  which  had  not  been  fully  matured,  has 
now  been  deferred  to  follow-on  development.  The 
next-generation  helmet  has  completed  initial 
testing  and  is  being  delivered  to  operators. 
Developmental  testing  is  progressing,  but,  with 
the  most  complex  testing  remaining,  more  design 
changes  are  likely.  Although  the  aircraft  designs 
were  not  stable  at  their  critical  design  reviews  in 
2006  and  2007,  all  baseline  engineering  drawings 
have  since  been  released.  Manufacturing 
efficiency  is  steady,  and  processes  are 
considered  in  control.  The  program  is  planning  a 
Block  4  upgrade  to  address  capability  deferrals, 
upgrade  existing  capabilities,  and  introduce 
additional  weapons. 


Page  163 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


Common  Name:  F-35 


F-35  Program 

Technology  and  Design  Maturity 

All  of  the  program's  critical  technologies  are 
considered  fully  mature.  One  technology  previously 
identified  as  critical  and  not  fully  matured,  the 
prognostics  and  health  management  system  a  part 
of  the  Autonomic  Logistics  Information  System 
(ALIS)  and  critical  to  fleet  operations,  has  been 
deferred  to  follow-on  development.  In  addition,  the 
prognostic  system  specific  to  the  engine  has  yet  to 
be  incorporated  into  ALIS.  The  program  has 
completed  initial  testing  of  the  next-generation 
helmet,  which  is  expected  to  enhance  night  vision 
and  optical  performance,  and  it  is  currently  being 
delivered  to  operators. 

Although  the  aircraft  designs  were  not  stable  at  their 
critical  design  reviews  in  2006  and  2007,  all 
baseline  engineering  drawings  have  since  been 
released.  The  program  continues  to  test  the  aircraft 
and  introduce  design  changes  to  address 
deficiencies  discovered  in  testing  concurrent  with 
production.  Design  changes  to  fix  issues  with  the 
bulkhead,  engine,  and  arresting  hook  have  been 
identified  and  planned  for  introduction  into 
production.  Although  the  total  number  of  design 
changes  generally  continues  to  fall,  with  the  most 
complex  developmental  testing  remaining,  the 
program  faces  the  risk  of  further  design  changes. 

Production  Maturity 

Aircraft  manufacturing  deliveries  remain  steady,  and 
the  contractor  has  delivered  154  aircraft  as  of 
December  2015.  Since  the  start  of  production,  the 
contractor's  production  processes  have  continued 
to  mature,  and  program  officials  stated  that  they  are 
now  at  a  manufacturing  readiness  level  that 
indicates  they  are  in  control.  To  continue  making 
improvements  and  to  increase  quality,  the 
contractor  tracks  statistical  process  control  data  and 
other  quality  indicators.  Production  part  shortages 
remain  a  risk  as  suppliers  will  face  additional 
pressures  of  balancing  an  increased  production  rate 
and  simultaneously  sustaining  a  growing 
operational  fleet. 

Other  Program  Issues 

In  July  2015,  the  Marine  Corps  declared  initial 
operational  capability  with  all  but  eight  of  the 
capabilities  expected  including  capabilities  centered 
on  sensor  fusion,  electronic  warfare,  and 


communication.  Marine  Corps  officials  stated  that 
these  shortcomings  do  not  interfere  with  their 
mission  set.  Program  officials  also  stated  that  these 
capabilities  will  be  available  for  the  initial  operational 
capability  of  the  Air  Force  and  Navy. 

In  August  201 5,  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  approved  the 
program’s  plan  for  follow-on  development, 
beginning  with  Block  4,  as  part  of  the  F-35’s  current 
baseline  development  program.  Block  4  is  expected 
to  consist  of  four  increments,  alternating  from 
primarily  software  in  the  first  and  third  increments  to 
hardware  and  software  upgrades  in  the  second  and 
fourth  increments.  Program  officials  stated  that 
Block  4  is  expected  to  enhance  capabilities  and 
introduce  new  weapons.  Block  4  will  also 
incorporate  capabilities  and  technologies  that  have 
been  deferred  from  the  baseline  program,  such  as 
the  prognostics  health  management  downlink 
capability.  The  program  estimates  that  Block  4 
development  will  take  about  10  years  to  complete. 

Program  Office  Comments 

In  addition  to  providing  technical  comments,  the 
program  office  noted  that  it  appreciates  GAO's 
reviews  in  assisting  the  program  by  identifying 
areas  for  improvement.  According  to  the  program 
office,  the  F-35  program  is  executing  well  across  the 
entire  spectrum  of  acquisition,  to  include 
development,  flight  test,  production,  fielding  and 
base  stand-up,  sustainment  of  fielded  aircraft,  and 
building  a  global  sustainment  enterprise.  The 
program  is  at  a  pivot  point  where  it  is  moving  from 
slow  and  steady  progress  to  a  rapidly  growing  and 
accelerating  program.  This  transition  is  not  without 
risks  and  challenges.  The  completion  of  mission 
systems  software  development,  ALIS  development, 
and  fuel  system  and  ejection  seat  deficiencies  are 
the  most  prominent,  current  technical  risks.  The 
ability  to  standup  four  separate  reprogramming  labs, 
and  our  ability  to  complete  all  weapons  envelope 
testing  for  Block  3F,  and  start  operational  test  on 
time,  constitute  major  schedule  risks.  The  program 
remains  confident  that  we  will  be  able  to  deliver  the 
full  F-35  capability  as  promised. 
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Lead  Component:  DOD-wide 


Common  Name:  JLTV 


Joint  Light  Tactical  Vehicle  (JLTV) 


The  Army  and  Marine  Corps'  JLTV  is  a  family  of 
vehicles  being  developed  to  replace  the  High 
Mobility  Multipurpose  Wheeled  Vehicle  (HMMWV) 
for  some  missions.  The  JLTV  is  expected  to  provide 
protection  for  passengers  against  current  and  future 
battlefield  threats,  increased  payload  capacity,  and 
improved  automotive  performance  over  the  up- 
armored  HMMWV.  It  must  also  be  transportable  by 
air  and  ship.  Two-  and  four-seat  variants  are 
planned  with  multiple  mission  configurations. 


Source:  JPO  JLTV  EMD  Industry  Contractors  (Oshkosh  Corp.,  AM  General  &  Lockheed  Martin). 
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Program  Essentials 

Program  Performance  (fiscal  year  2016  dollars  in  millions) 

Prime  contractor:  AM  General, 

As  of 

Latest 

Percent 

Lockheed  Martin,  Oshkosh 

10/2012 

06/2015 

change 

Program  office:  Harrison 

Research  and  development  cost 

$1,021.3 

$993.3 

-2.7% 

Township,  Ml 

Procurement  cost 

$23,118.2 

$22,962.7 

-0.7% 

Funding  needed  to  complete: 

Total  program  cost 

$24,177.7 

$23,956.1 

-0.9% 

R&D:  $210.8  million 

Program  unit  cost 

$0,442 

$0,438 

-0.9% 

Procurement:  $22,793.6  million 

Total  quantities 

54,730 

54,718 

-0.0% 

Total  funding:  $23,004.4  million 

Acquisition  cycle  time  (months) 

125 

144 

15.2% 

Procurement  quantity:  54,408 

Both  JLTV  critical  technologies  are  fully  mature 
and,  according  to  officials,  have  been  integrated 
and  tested  on  production-representative  vehicles. 
The  government  conducted  design  understanding 
reviews  for  all  three  competing  vehicle  designs  to 
assess  their  technical  baselines.  The  Joint 
Program  Office  also  conducted  production 
readiness  reviews  for  all  three  contractors,  and, 
according  to  officials,  ultimately  determined  all 
three  were  qualified  for  production.  In  August 
2015,  the  Linder  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  approved 
JLTV  for  production  and  deployment.  That  same 
month,  the  Army  awarded  a  production  contract 
to  Oshkosh  Defense.  Contract  performance  was 
delayed  after  one  of  the  losing  vendors  filed  bid 
protests,  which  were  dismissed  in  December 
2015. 
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Common  Name:  JLTV 


JLTV  Program 

Technology  and  Design  Maturity 

According  to  the  program  office,  its  two  critical 
technologies-underbelly  protection  armor  and  side- 
kit  armor-are  fully  mature.  In  May  2014,  officials 
responsible  for  JLTV  requirements  oversight 
recommended  that  the  two  different  protection 
levels  desired  by  the  Army  and  Marine  Corps  be 
reduced  to  one  requirement  and  addressed  with  a 
single  add-on  armor  configuration.  As  a  result, 
program  officials  said  the  Army  and  Marine  Corps 
will  now  both  use  the  same  higher  protection  level 
configuration.  Program  officials  anticipate  this 
change  will  result  in  over  $12  million  in  live  fire  test 
and  evaluation  cost  avoidance.  According  to  Army 
officials,  prototype  systems  with  the  two  armor 
critical  technologies  have  been  tested  in  a  realistic 
environment,  although  results  were  not  provided  for 
our  review.  The  Army  has  declared  both 
technologies  mature  and  demonstrated  under 
operational  conditions. 

The  program  office  did  not  hold  a  formal  critical 
design  review  during  development  and  instead 
conducted  design  understanding  reviews  with 
contractors  between  December  2012  and  January 
2013.  According  to  program  officials,  these  reviews 
were  at  a  level  of  detail  similar  to  a  critical  design 
review  and  verified  that  all  contractors  had  more 
than  90  percent  of  the  design  files  under 
configuration  control. 

Production  Maturity 

According  to  the  Army,  the  low  numbers  of  vehicles 
built  during  system  development  would  not  allow  for 
statistically  relevant  measurements  to  demonstrate 
the  control  of  manufacturing  processes,  such  as  the 
use  of  process  capability  index  data.  The  production 
contractor  may  use  this  type  of  data  as  part  of  its 
quality  management,  but  according  to  the  program 
office  there  is  currently  no  contractual  requirement 
to  do  so.  To  assess  production  readiness  and 
manufacturing  risks  prior  to  production  start,  the 
program  instead  conducted  a  manufacturing 
readiness  assessment  using  manufacturing 
readiness  levels.  In  August  and  September  of  2014, 
the  program  held  production  readiness  reviews  for 
the  three  competing  contractors  to  examine 
manufacturing  process  readiness,  quality 
management  systems,  and  production  planning.  All 
competing  contractors  built  fully  configured 


production-representative  prototypes  on  pilot 
production  lines.  According  to  Army  officials,  these 
production-representative  prototypes  of  all  three 
vehicle  variants  were  subjected  to  developmental, 
operational,  and  live  fire  testing.  Based  on  those 
reviews  and  the  pilot  production  results,  the  joint 
program  office  determined  that  all  three  contractors 
demonstrated  proven  manufacturing  processes  and 
procedures.  While  the  program's  manufacturing 
process  maturity  may  have  reached  DOD's 
recommended  level  for  production,  it  has  not 
reached  the  level  that  indicates  processes  are  in 
control  as  recommended  by  GAO  best  practices. 

Other  Program  Issues 

In  August  201 5,  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  approved 
JLTV  for  production  and  deployment.  That  same 
month,  the  Army  awarded  a  production  contract  to 
Oshkosh  Defense.  Following  the  award,  one  of  the 
losing  vendors,  Lockheed  Martin  Corporation,  filed 
bid  protests  with  GAO,  which  were  dismissed  in 
December  2015.  Performance  of  the  awarded 
contract  was  suspended  pending  resolution  of 
issues  raised  in  bid  protests. 

Program  Office  Comments 

The  program  office  agreed  with  the  assessment  and 
provided  technical  comments  that  were 
incorporated  where  appropriate. 
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Agency  Comments 
and  Our  Evaluation 


We  are  not  making  recommendations  in  this  report.  DOD  provided  written 
comments  on  a  draft  of  this  report.  The  comments  are  reprinted  in 
appendix  VII.  We  also  received  technical  comments  from  DOD,  which 
were  incorporated  as  appropriate. 

In  its  comments,  DOD  noted  that  it  was  pleased  that  our  report  identifies 
ways  in  which  the  department  continues  to  drive  down  the  cost  of  the 
acquisition  portfolio.  DOD  attributes  this  decrease  in  overall  cost  to  the 
employment  of  the  “Better  Buying  Power”  initiatives.  The  department  also 
provided  a  chart  tracking  the  5-year  moving  average  of  annual  growth  in 
contract  costs  over  the  past  30  years,  a  measure  not  used  in  our  analysis, 
which  shows  a  decline  in  such  growth  following  the  implementation  of 
“Better  Buying  Power.”  We  have  not  included  this  chart  in  the  appendix  as 
there  could  be  other  explanations  for  this  decrease. 


We  are  sending  copies  of  this  report  to  interested  congressional 
committees  and  offices;  the  Secretary  of  Defense;  the  Secretaries  of  the 
Army,  Navy,  and  Air  Force;  and  the  Director  of  the  Office  of  Management 
and  Budget.  In  addition,  the  report  will  be  made  available  at  no  charge  on 
the  GAO  Web  site  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (202)  512-4841 .  Contact  points  for  our  offices  of 
Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last  page 
of  this  report.  Staff  members  making  key  contributions  to  this  report  are 
listed  in  appendix  VIII. 


Michael  J.  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
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List  of  Committees 


The  Honorable  John  McCain 
Chairman 

The  Honorable  Jack  Reed 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Thad  Cochran 
Chairman 

The  Honorable  Richard  J.  Durbin 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Mac  Thornberry 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Rodney  Frelinghuysen 
Chairman 

The  Honorable  Pete  Visclosky 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Appendix  I:  Scope  and  Methodology 


Analysis  of  the  Cost 
Performance  of 
DOD’s  Portfolio  of 
Major  Defense 
Acquisition  Programs 


To  develop  our  11  observations  on  the  size,  cost,  and  schedule  of 
Department  of  Defense’s  (DOD)  portfolio  of  current  major  defense 
acquisition  programs,  we  obtained  and  analyzed  cost,  quantity,  and 
schedule  data  from  Selected  Acquisition  Reports  (SAR)  and  other 
information  in  the  Defense  Acquisition  Management  Information  Retrieval 
Purview  system,  referred  to  as  DAMIR.1  We  entered  this  data  into  a 
database  and  verified  that  the  data  was  entered  correctly.  We  converted 
all  cost  information  to  fiscal  year  2016  dollars  using  conversion  factors 
from  the  DOD  Comptroller’s  National  Defense  Budget  Estimates  for 
Fiscal  Year  2016  (table  5-9).  To  assess  the  reliability  of  the  SAR  data  we 
talked  with  a  DOD  official  responsible  for  DAMIR’s  data  quality  control 
procedures  and  reviewed  relevant  documentation.  We  also  confirmed 
selected  data  reliability  with  program  offices.  We  determined  that  the  SAR 
data  and  the  information  retrieved  from  DAMIR  were  sufficiently  reliable 
for  the  purposes  of  this  report.  Our  assessment  includes  comparisons  of 
size,  cost,  and  schedule  changes  over  the  past  year,  five  years,  and  from 
baseline  estimates  that  utilize  SAR  data  from  December  2014,  December 
2013,  December  2009,  and  from  the  programs’  initial  SAR  submissions. 
We  also  analyzed  the  data  to  determine  the  number  of  programs  in  each 
portfolio  year.  In  general,  we  refer  to  the  79  major  defense  acquisition 
programs  with  SARs  dated  December  2014  as  DOD’s  2015  or  current 
portfolio  and  use  a  similar  convention  for  prior  year  portfolios.  We 
retrieved  data  on  research,  development,  test,  and  evaluation; 
procurement;  military  construction,  acquisition  operation  and 
maintenance,  total  acquisition  cost,  and  schedule  estimates  for  the  79 
programs  in  the  2015  portfolio.2 


We  divided  two  programs  into  two  distinct  elements,  because  DOD 
reports  performance  data  on  them  separately.  As  a  result  some  of  our 
analysis  reflects  a  total  of  81  programs  and  sub-elements.  The  Missile 
Defense  Agency’s  Ballistic  Missile  Defense  System  is  excluded  from  all 
analyses  as  the  program  does  not  have  an  integrated  long-term  baseline, 


^AMIR  Purview  is  an  executive  information  system  operated  by  the  Office  of  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  /  Acquisition  Resources 
and  Analysis. 

2We  refer  to  research,  development,  test,  and  evaluation  costs  as  research  and 
development  or  simply  as  development  costs  in  this  report.  Total  acquisition  cost  includes 
research  and  development  and  procurement  costs  as  well  as  acquisition  related  operation 
and  maintenance  and  system-specific  military  construction  costs. 
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which  prevents  us  from  assessing  the  program’s  cost  progress  or 
comparing  it  to  other  major  defense  acquisition  programs. 

For  our  first  observation,  we  compared  the  2015  portfolio  with  the 
programs  that  issued  SARs  in  December  2013  (2014  portfolio)  to  identify 
the  programs  that  exited  and  entered  the  current  portfolio.  We  accounted 
for  their  cost  as  well  as  the  difference  in  cost  between  the  two  portfolios. 

For  the  second  observation,  we  aggregated  funding  stream  data  for  the 
total  planned  investment  of  each  portfolio  from  DAMIR  for  each  year 
since  2005  to  determine  any  trends.  We  determined  the  yearly  totals  for 
research  and  development,  procurement,  and  total  acquisition  cost.  To 
distinguish  the  funding  already  invested  from  the  funding  remaining  that  is 
needed  to  complete  the  programs  in  each  portfolio  since  2005,  we  used 
funding  stream  data  obtained  from  DAMIR  for  each  December  SAR 
submission  for  the  years  2004  (2005  portfolio)  through  2014  (2015 
portfolio).  We  define  funding  invested  as  all  funding  that  has  been 
provided  to  the  programs  in  the  fiscal  year  of  the  annual  SAR  submission 
(this  includes  fiscal  year  2015  for  the  December  2014  submission)  and 
earlier,  while  funding  remaining  is  all  the  amounts  that  DOD  anticipates 
will  be  needed  in  the  fiscal  years  after  the  annual  SAR  submission  (fiscal 
year  2016  and  later  for  the  December  2014  submission). 

For  our  third  observation,  we  determined  the  cost  and  schedule  changes 
on  defense  acquisition  programs  in  the  current  portfolio  over  the  past 
year,  five  years,  and  from  baseline  estimates.  To  do  this,  we  collected 
data  from  December  2014,  December  2013,  December  2009,  and  from 
programs’  initial  SARs;  acquisition  program  baselines;  and  program 
offices. 

For  programs  less  than  a  year  old,  we  calculated  the  difference  between 
the  December  2014  SAR  current  estimate  and  the  first  full  estimate  in 
order  to  identify  the  cost  and  schedule  change  over  the  past  year.  For 
programs  less  than  five  years  old,  we  took  a  similar  approach  when 
calculating  the  cost  and  schedule  change  over  the  past  five  years. 

For  the  fourth  observation  we  used  SAR  data  to  determine  which  10 
programs  had  the  largest  total  acquisition  cost,  their  age,  and  the  total 
cost  and  cost  growth  since  first  full  estimates  for  two  groups  of 
programs — programs  started  in  the  past  5  years  and  program  started  5  or 
more  years  ago. 
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For  our  fifth  observation,  we  divided  the  programs  into  percent  cost 
change  categories  based  on  the  percent  change  in  total  acquisition  cost 
they  experienced  over  the  past  year.  We  then  totaled  the  number  of 
programs  in  each  category  and  the  total  cost  change  of  the  programs  in 
each  category. 

For  our  sixth  observation,  in  order  to  determine  whether  programs 
experienced  an  increase  or  decrease  in  buying  power  over  the  past  year 
and  five  years,  we  used  data  on  the  programs’  number  of  procurement 
units,  procurement  cost  changes,  and  average  procurement  unit  costs. 

We  first  calculated  the  amount  of  procurement  cost  growth  attributable  to 
quantity  changes.  To  do  this,  we  multiplied  any  change  in  quantity  by  the 
average  procurement  unit  cost  for  the  program  a  year  ago. 

The  resulting  dollar  amount  is  considered  a  change  due  solely  to  shifts  in 
the  number  of  units  procured  and  may  overestimate  the  amount  of 
change  expected  when  quantities  increase  and  underestimate  the 
expected  change  when  quantities  decrease  as  it  does  not  account  for 
other  effects  of  quantity  changes  on  procurement  such  as  gain  or  loss  of 
learning  in  production  that  could  result  in  changes  to  unit  cost  over  time  or 
the  use  or  absence  of  economic  orders  of  material.  However,  these 
changes  are  accounted  for  as  part  of  the  change  in  cost  not  due  to 
quantities.  To  determine  whether  any  trends  in  buying  power  changes 
occurred  over  the  past  five  years,  once  we  conducted  the  calculations  just 
described,  we  excluded  any  programs  that  were  not  in  each  of  the  past 
five  portfolios.  As  a  result,  we  examined  61  major  defense  acquisition 
programs  for  their  buying  power  increases  or  decreases  over  the  past  five 
years. 

For  our  seventh  observation,  we  used  schedule  information  from  the 
SARs  and  calculated  the  acquisition  cycle  time  from  program 
development  start  to  initial  operational  capability  and  any  delay  in 
achieving  initial  operational  capability.  For  programs  that  do  not  have  a 
development  estimate,  we  compared  the  current  estimate  to  the 
production  estimate.  For  shipbuilding  programs  with  a  planning  estimate, 
we  compared  the  current  estimate  to  the  planning  estimate.  For  programs 
that  began  as  pre-major  defense  acquisition  programs,  the  first  full 
estimate  we  used  as  a  cost  and  schedule  baseline  may  differ  from  the 
baseline  disclosed  in  the  program’s  initial  DOD  SAR  submission. 

For  programs  in  the  current  portfolio  where  schedule  data  for  initial 
operational  capability  was  not  available  over  the  past  year  and  five  years, 
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we  used  a  similar  methodology  as  used  when  calculating  cost  change  for 
programs  that  are  less  than  a  year  old  and  less  than  five  years  old. 

For  our  eighth  observation,  we  determined  which  programs  reported  a 
development  cost  estimate  increase  more  than  2  percent  over  the  past 
year  and  determined  how  many  of  these  were  in  production. 

For  the  ninth  observation,  we  evaluated  program  performance  against 
high-risk  criteria  discussed  by  DOD,  the  Office  of  Management  and 
Budget  (OMB),  and  GAO.  We  calculated  how  many  programs  had  less 
than  a  2  percent  increase  in  total  acquisition  cost  over  the  past  year,  less 
than  a  10  percent  increase  over  the  past  five  years,  and  less  than  a  15 
percent  increase  from  baseline  estimates  using  data  from  SARs;  initial 
acquisition  program  baselines;  and  program  offices.  We  calculated  the 
percentage  of  programs  meeting  each  of  these  high-risk  criteria  for  the 
2011-2015  portfolios  to  identify  any  changes. 

For  programs  with  multiple  sub-programs  presented  in  the  SARs  we 
calculated  the  net  effect  of  the  sub-programs  to  reach  an  aggregate 
program  result. 

For  our  10th  observation  we  used  funding  stream  data  from  SARs  to 
analyze  the  percent  of  total  acquisition  cost  in  the  2005-2015  portfolios 
that  is  accounted  for  by  each  of  the  services.  Where  the  Department  of 
Defense  is  identified  as  the  lead  component,  we  allocated  the  service- 
specific  portion  of  total  acquisition  costs:  for  the  F-35  Joint  Strike  Fighter, 
to  the  Air  Force  and  the  Navy;  and  for  the  Joint  Light  Tactical  Vehicle,  to 
the  Army  and  Navy.  The  Chemical  Demilitarization  program,  where  DOD 
is  identified  as  the  lead,  does  not  have  service-specific  funding. 

For  the  1 1th  observation,  we  used  SAR  data  to  determine  which  10 
programs  had  the  largest  total  acquisition  cost,  and  the  prime  contractors 
associated  with  these  programs.  We  gathered  information  from 
Bloomberg  on  the  equity  prices  of  each  of  these  contractors  from  2002  to 
2014,  and  compared  the  returns  from  these  equities  to  the  S&P  500  and 
the  Industrials  sector,  as  defined  by  the  Standard  and  Poor’s  Global 
Industry  Classification  Standard. 


Analysis  of  Selected  DOD 
Programs  Using 
Knowledge-Based  Criteria 


To  collect  data  from  current  and  future  major  defense  acquisition 
programs — including  cost  and  schedule  estimates,  technology  maturity, 
and  planned  implementation  of  acquisition  reforms — we  distributed  one 
data  collection  instrument  and  two  electronic  questionnaires,  one 
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questionnaire  for  the  43  current  programs  and  a  slightly  different 
questionnaire  for  the  12  future  programs.  Both  of  the  questionnaires  were 
sent  by  e-mail  in  an  attached  Microsoft  Word  form  that  respondents  could 
return  electronically.  We  received  responses  from  all  of  the  programs  we 
assessed  from  August  to  September  2015.  To  ensure  the  reliability  of  the 
data  collected  through  the  data  collection  instrument  and  our 
questionnaires,  we  took  a  number  of  steps  to  reduce  measurement  error 
and  non-response  error. 

These  steps  included  conducting  three  pretests  of  the  future  major 
defense  acquisition  program  questionnaire  and  three  pretests  for  the 
current  major  defense  acquisition  program  questionnaire  prior  to 
distribution  to  ensure  that  our  questions  were  clear,  unbiased,  and 
consistently  interpreted;  reviewing  responses  to  identify  obvious  errors  or 
inconsistencies;  cross-referencing  information  provided  in  the  data 
collection  instrument  with  the  questionnaire;  conducting  follow-up  to 
clarify  responses  when  needed;  and  verifying  the  accuracy  of  a  sample  of 
keypunched  questionnaires.  Our  pretests  covered  each  branch  of  the 
military  to  better  ensure  that  the  questionnaires  could  be  understood  by 
officials  within  each  branch.  We  determined  that  the  data  were  sufficiently 
reliable  for  the  purposes  of  this  report. 

Our  analysis  of  how  well  programs  are  adhering  to  a  knowledge-based 
acquisition  approach  focuses  on  43  major  defense  acquisition  programs 
that  are  mostly  in  development  or  the  early  stages  of  production.  To 
assess  the  knowledge  attained  by  key  decision  points  (system 
development  start  or  detailed  design  contract  award  for  shipbuilding 
programs,  critical  design  review  or  fabrication  start  for  shipbuilding 
programs,  and  production  start),  we  collected  data  from  program  offices 
about  their  knowledge  at  each  point.  In  particular,  we  focused  on  the 
seventeen  programs  that  crossed  these  key  acquisition  points  in  2015  or 
planned  to  in  early  2016  and  evaluated  their  adherence  to  knowledge 
based  practices. 

We  also  provide  information  on  how  much  knowledge  is  obtained  at  key 
decision  points  by  programs  that  accomplished  these  previously.  We  also 
included  observations  on  the  knowledge  that  12  future  programs  expect 
to  obtain  before  starting  development.  We  did  not  validate  the  data 
provided  by  the  program  offices,  but  reviewed  the  data  and  performed 
various  checks  to  determine  that  they  were  reliable  enough  for  our 
purposes.  Where  we  discovered  discrepancies,  we  clarified  the  data 
accordingly. 
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The  55  current  and  future  programs  included  in  our  assessment  were  in 
various  stages  of  the  acquisition  cycle,  and  not  all  of  the  programs 
provided  information  on  knowledge  obtained  at  each  point.  Programs 
were  not  included  in  our  assessments  at  key  decision  points  if  relevant 
data  were  not  available.  Our  analysis  of  knowledge  attained  at  each  key 
point  includes  factors  that  we  have  previously  identified  as  being  key  to  a 
knowledge-based  acquisition  approach,  including  holding  early  systems 
engineering  reviews,  testing  an  integrated  prototype  prior  to  the  design 
review,  using  a  reliability  growth  curve,  planning  for  manufacturing,  and 
testing  a  production-representative  prototype  prior  to  the  making  a 
production  decision.  Additional  information  on  how  we  collect  these  data 
is  found  in  the  product  knowledge  assessment  section  of  appendix  I.  See 
also  appendix  IV  for  a  list  of  the  practices  that  are  associated  with  a 
knowledge-based  acquisition  approach. 


Analysis  of  Acquisition 
Initiatives  and  Program 
Concurrency 


To  develop  observations  on  how  DOD  is  implementing  acquisition 
reforms,  we  reviewed  the  DOD  Instruction  5000.02,  the  Weapon  Systems 
Acquisition  Reform  Act  of  2009  (WSARA),  and  the  September  19,  2014, 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
“Better  Buying  Power  3.0  Interim  Release.”  and  previous  releases  of 
DOD’s  “Better  Buying  Power”  memoranda.3  We  analyzed  questionnaire 
data  received  from  the  43  current  and  12  future  major  defense  acquisition 
programs  in  our  assessment  to  determine  the  extent  to  which  acquisition 
reforms  have  been  implemented.  We  determined  which  programs  have 
established  affordability  constraints  and,  for  current  programs,  examined 
the  median  development  cost  growth  on  programs  with  these  constraints 
compared  to  those  without.  We  tallied  programs  that  conducted  “should- 
cost”  analyses  and  identified  realized  and/or  future  potential  savings.  We 
also  analyzed  whether  programs  are  planning  for  competition  throughout 
the  acquisition  life-cycle. 

To  examine  programs’  software  development  efforts  we  identified  the 
programs  that  reported  their  software  as  high-risk.  We  used  the 
questionnaire  responses  from  these  programs  to  assess  the  reasons  why 
they  identified  their  software  effort  as  high-risk  and  the  metrics  programs 
reported  using  to  manage  their  software  development.  We  identified  the 
dates  reported  by  programs  for  their  software  and  hardware  integration 


3Pub.  L.  No.  111-23. 
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Individual  Assessments  of 
Weapon  Programs 


and  compared  those  dates  to  each  program’s  production  start  date  to 
assess  each  programs’  degree  of  software  development  and  production 
concurrency.  We  also  determined  the  average  difference  in  schedule 
delay  for  programs  that  identified  their  software  development  as  high  risk 
versus  those  that  did  not. 

To  assess  program  development  testing  and  production  concurrency  we 
identified  the  programs — among  those  we  included  in  our  assessment — 
with  production  start  dates.  We  used  the  questionnaire  responses  from 
those  programs  to  identify  the  dates  for  the  start  and  end  of 
developmental  testing,  compared  those  dates  to  the  timing  of  each 
program’s  production  decision  and  determined  the  number  of  months,  if 
any,  of  developmental  testing  done  after  production  start.  We  then 
compared  the  number  of  overlapping  months  to  the  total  number  of 
months  of  developmental  testing  for  each  program  and  calculated  the 
percentage  of  developmental  testing  done  concurrent  with  production. 
Lastly,  we  analyzed  the  correlation  between  concurrency  and  total 
acquisition  cost  growth  for  programs  in  production,  and  the  correlation 
between  total  acquisition  cost  and  concurrency. 

In  total,  this  report  presents  individual  assessments  of  55  weapon 
programs.  A  table  listing  these  programs  is  found  in  appendix  VIII.  Out  of 
these  programs,  42  are  captured  in  a  two-page  format  discussing 
technology,  design,  and  manufacturing  knowledge  obtained  and  other 
program  issues.  Thirty-nine  of  these  42  two-page  assessments  are  of 
major  defense  acquisition  programs,  most  of  which  are  in  development  or 
early  production  and  three  assessments  are  of  programs  that  were 
projected  to  become  major  defense  acquisition  programs  during  or  soon 
after  our  review.  The  remaining  13  programs  are  described  in  a  one-page 
format  that  describes  their  current  status.  Those  one-page  assessments 
include  12  future  major  defense  acquisition  programs  and  one  major 
defense  acquisition  program  that  is  well  into  production. 

For  presentation  purposes  we  grouped  the  individual  assessments  by 
lead  service — Army,  Navy  and  Marine  Corps,  Air  Force,  and  DOD-led — 
and  inserted  a  lead  service  separator  page  at  the  start  of  each  grouping. 
These  four  separator  pages  summarize  information  about  the  acquisition 
phase,  current  estimated  funding  needs,  cost  and  schedule  growth,  and 
product  knowledge  attained  that  is  provided  in  the  one  and  two-page 
assessments.  We  report  cost  and  schedule  growth  in  the  separator  pages 
in  a  manner  that  is  consistent  with  how  it  is  reported  and  described 
elsewhere  in  the  report.  Estimates  of  funding  needed  to  complete  in  the 
separator  pages  are  based  on  all  amounts  that  will  be  provided  in  fiscal 


Page  175 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


Appendix  I:  Scope  and  Methodology 


year  2016  and  later.  For  some  future  major  defense  acquisition  programs 
the  estimates  of  funding  needed  represents  only  those  amounts  provided 
through  fiscal  year  2020  and  are  not  a  full  to  complete  amount. 

Over  the  past  several  years,  DOD  has  revised  policies  governing  weapon 
system  acquisitions  and  changed  the  terminology  used  for  major 
acquisition  events.  To  make  DOD’s  acquisition  terminology  more 
consistent  across  the  55  program  assessments,  we  standardized  the 
terminology  for  key  program  events.  For  most  individual  programs  in  our 
assessment,  “development  start”  refers  to  the  initiation  of  an  acquisition 
program  as  well  as  the  start  of  engineering  and  manufacturing 
development  or  system  development.  This  generally  coincides  with 
DOD’s  milestone  B.  A  few  programs  in  our  assessment  have  a  separate 
“program  start”  date,  which  begins  a  pre-system  development  phase  for 
program  definition  and  risk-reduction  activities.  This  “program  start”  date 
generally  coincides  with  DOD’s  former  terminology  for  milestone  I  or 
DOD’s  current  milestone  A,  which  denotes  the  start  of  technology 
maturation  and  risk  reduction.  The  “production  decision”  generally  refers 
to  the  decision  to  enter  the  production  and  deployment  phase,  typically 
with  low-rate  initial  production.  The  “initial  capability”  refers  to  the  initial 
operational  capability — sometimes  called  first  unit  equipped  or  required 
asset  availability.  For  shipbuilding  programs,  the  schedule  of  key  program 
events  in  relation  to  acquisition  milestones  varies  for  each  program.  Our 
work  on  shipbuilding  best  practices  has  identified  the  detailed  design 
contract  award  and  the  start  of  lead  ship  fabrication  as  the  points  in  the 
acquisition  process  roughly  equivalent  to  development  start  and  design 
review  for  other  programs. 

For  each  program  we  assessed  in  a  two-page  format,  we  present  cost, 
schedule,  and  quantity  data  at  the  program’s  first  full  estimate  and  an 
estimate  from  the  latest  SAR  or  the  program  office  reflecting  2015  data 
where  they  were  available.  The  first  full  estimate  is  generally  the  cost 
estimate  established  at  milestone  B — development  start;  however,  for  a 
few  programs  that  did  not  have  such  an  estimate,  we  used  the  estimate  at 
milestone  C — production  start — instead.  For  shipbuilding  programs,  we 
used  their  planning  estimates  if  those  estimates  were  available.  For 
systems  for  which  a  first  full  estimate  was  not  available,  we  only  present 
the  latest  available  estimate  of  cost  and  quantities.  For  the  other 
programs  assessed  in  a  one-page  format,  we  present  the  latest  available 
estimate  of  cost  and  quantity  from  the  program  office. 

For  each  program  we  assessed,  all  cost  information  is  presented  in  fiscal 
year  2016  dollars.  We  converted  cost  information  to  fiscal  year  2016 
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dollars  using  conversion  factors  from  the  DOD  Comptroller’s  National 
Defense  Budget  Estimates  for  Fiscal  Year  2016  (table  5-9).  We  have 
depicted  only  the  program’s  main  elements  of  acquisition  cost — research 
and  development  and  procurement.  However,  the  total  program  cost  also 
includes  military  construction  and  acquisition-related  operation  and 
maintenance  costs.  Because  of  rounding  and  these  additional  costs,  in 
some  situations,  total  cost  may  not  match  the  exact  sum  of  the  research 
and  development  and  procurement  costs.  The  program  unit  costs  are 
calculated  by  dividing  the  total  program  cost  by  the  total  quantities 
planned.  In  some  instances,  the  data  were  not  applicable,  and  we 
annotate  this  by  using  the  term  “not  applicable  (NA).”  The  quantities  listed 
refer  to  total  quantities,  including  both  procurement  and  development 
quantities. 

The  schedule  assessment  for  each  program  is  based  on  acquisition  cycle 
time,  defined  as  the  number  of  months  between  program  start  and  the 
achievement  of  initial  operational  capability  or  an  equivalent  fielding  date. 
In  some  instances  the  data  were  not  yet  available,  and  we  annotate  this 
by  using  the  term  “to  be  determined  (TBD)”  or  “NA.” 

The  information  presented  on  the  “funding  needed  to  complete”  is  from 
fiscal  year  2016  through  completion  and,  unless  otherwise  noted,  draws 
on  information  from  SARs  or  on  data  from  the  program  office.  In  some 
instances,  the  data  were  not  available,  and  we  annotate  this  by  the  term 
“TBD”  or  “NA.”  The  quantities  listed  refer  only  to  procurement  quantities. 
Satellite  programs,  in  particular,  produce  a  large  percentage  of  their  total 
operational  units  as  development  quantities,  which  are  not  included  in  the 
quantity  figure. 

The  intent  of  these  comparisons  is  to  provide  an  aggregate,  or  overall, 
picture  of  a  program’s  history.  These  assessments  represent  the  sum  of 
the  federal  government’s  actions  on  a  program,  not  just  those  of  the 
program  manager  and  the  contractor.  DOD  does  a  number  of  detailed 
analyses  of  changes  that  attempt  to  link  specific  changes  with  triggering 
events  or  causes.  Our  analysis  does  not  attempt  to  make  such  detailed 
distinctions. 

In  this  year’s  assessment  we  also  reviewed  whether  individual 
subcontracting  reports  from  a  program’s  prime  contractor  or  contractors 
were  accepted  on  the  Electronic  Subcontracting  Reporting  System 
(eSRS).  We  reviewed  this  information  for  79  of  the  major  defense 
acquisition  programs  included  in  our  assessment  using  the  contract 
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Product  Knowledge  Data  on 
Individual  Two-Page 
Assessments 


information  reported  in  their  December  2014  Selected  Acquisition 
Reports.  See  appendix  VI  for  a  list  of  the  programs  we  reviewed. 

In  our  past  work  examining  weapon  acquisition  issues  and  best  practices 
for  product  development,  we  have  found  that  leading  commercial  firms 
pursue  an  acquisition  approach  that  is  anchored  in  knowledge,  whereby 
high  levels  of  product  knowledge  are  demonstrated  by  critical  points  in 
the  acquisition  process.  On  the  basis  of  this  work,  we  have  identified 
three  key  knowledge  points  during  the  acquisition  cycle — system 
development  start,  critical  design  review,  and  production  start — at  which 
programs  need  to  demonstrate  critical  levels  of  knowledge  to  proceed.  To 
assess  the  product  development  knowledge  of  each  program  at  these  key 
points,  we  reviewed  data-collection  instruments  and  questionnaires 
submitted  by  programs;  however,  not  every  program  had  responses  to 
each  element  of  the  data-collection  instrument  or  questionnaire.  We  also 
reviewed  pertinent  program  documentation  and  discussed  the  information 
presented  on  the  data-collection  instrument  and  questionnaire  with 
program  officials  as  necessary. 

To  assess  a  program’s  readiness  to  enter  system  development,  we 
collected  data  through  the  data-collection  instrument  on  critical 
technologies  and  early  design  reviews.  To  assess  technology  maturity, 
we  asked  program  officials  to  apply  a  tool,  referred  to  as  technology 
readiness  levels  (TRL),  for  our  analysis.  The  National  Aeronautics  and 
Space  Administration  originally  developed  TRLs,  and  the  Army  and  Air 
Force  science  and  technology  research  organizations  use  them  to 
determine  when  technologies  are  ready  to  be  handed  off  from  science 
and  technology  managers  to  product  developers.  TRLs  are  measured  on 
a  scale  from  1  to  9,  beginning  with  paper  studies  of  a  technology’s 
feasibility  and  culminating  with  a  technology  fully  integrated  into  a 
completed  product.  See  appendix  V  for  TRL  definitions.  Our  best- 
practices  work  has  shown  that  a  TRL  7 — demonstration  of  a  technology  in 
an  operational  environment — is  the  level  of  technology  maturity  that 
constitutes  a  low  risk  for  starting  a  product  development  program.4  For 
shipbuilding  programs,  we  have  recommended  that  this  level  of  maturity 


4GAO,  Best  Practices:  Better  Management  of  Technology  Development  Can  Improve 
Weapon  System  Outcomes ,  GAO/NSIAD-99-162  (Washington,  D.C.:  July  30,  1999); 
GAO,  Best  Practices:  Better  Matching  of  Needs  and  Resources  Will  Lead  to  Better 
Weapon  System  Outcomes,  GAO-01-288  (Washington,  D.C.:  Mar.  8,  2001). 


Page  178 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


Appendix  I:  Scope  and  Methodology 


be  achieved  by  the  contract  award  for  detailed  design.5  In  our 
assessment,  the  technologies  that  have  reached  TRL  7,  a  prototype 
demonstrated  in  an  operational  environment,  are  referred  to  as  mature  or 
fully  mature.  Those  technologies  that  have  reached  TRL  6,  a  prototype 
demonstrated  in  a  relevant  environment,  are  referred  to  as  approaching 
or  nearing  maturity.  Satellite  technologies  that  have  achieved  TRL  6  are 
assessed  as  fully  mature  due  to  the  difficulty  of  demonstrating  maturity  in 
an  operational  environment — space.  In  addition,  we  asked  program 
officials  to  provide  the  date  of  the  preliminary  design  review.  We 
compared  this  date  to  the  system  development  start  date. 

In  most  cases,  we  did  not  validate  the  program  offices’  selection  of  critical 
technologies  or  the  determination  of  the  demonstrated  level  of  maturity. 
We  sought  to  clarify  the  TRLs  in  those  cases  where  information  existed 
that  raised  concerns.  If  we  were  to  conduct  a  detailed  review,  we  might 
adjust  the  critical  technologies  assessed,  their  readiness  levels 
demonstrated,  or  both.  It  was  not  always  possible  to  reconstruct  the 
technological  maturity  of  a  weapon  system  at  key  decision  points  after  the 
passage  of  many  years.  Where  practicable,  we  compared  technology 
assessments  provided  by  the  program  office  to  assessments  conducted 
by  officials  from  the  Office  of  the  Assistant  Secretary  of  Defense  for 
Research  and  Engineering. 

To  assess  design  stability,  we  asked  program  officials  to  provide  the 
percentage  of  design  drawings  completed  or  projected  for  completion  by 
the  design  review,  the  production  decision,  and  as  of  our  current 
assessment  in  the  data-collection  instrument.  In  most  cases,  we  did  not 
verify  or  validate  the  percentage  of  engineering  drawings  provided  by  the 
program  office.  We  clarified  the  percentage  of  drawings  completed  in 
those  cases  where  information  that  raised  concerns  existed.  Completed 
drawings  were  defined  as  the  number  of  drawings  released  or  deemed 
releasable  to  manufacturing  that  can  be  considered  the  “build  to” 
drawings.  For  shipbuilding  programs,  we  asked  program  officials  to 
provide  the  percentage  of  the  three-dimensional  product  model  that  had 
been  completed  by  the  start  of  lead  ship  fabrication,  and  as  of  our  current 
assessment.  To  gain  greater  insights  into  design  stability,  we  also  asked 
program  officials  to  provide  the  date  they  planned  to  first  integrate  and 


5GAO,  Best  Practices:  High  Levels  of  Knowledge  at  Key  Points  Differentiate  Commercial 
Shipbuilding  from  Navy  Shipbuilding,  GAO-09- 322  (Washington,  D.C.:  May  13,  2009). 
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test  all  key  subsystems  and  components  into  a  system-level  integrated 
prototype.  We  compared  this  date  to  the  date  of  the  design  review.  We 
did  not  assess  whether  shipbuilding  programs  had  completed  integrated 
prototypes. 

To  assess  production  maturity,  we  asked  program  officials  to  identify  the 
number  of  critical  manufacturing  processes  and,  where  available,  to 
quantify  the  extent  of  statistical  control  achieved  for  those  processes  as  a 
part  of  our  data-collection  instrument.  In  most  cases,  we  did  not  verify  or 
validate  the  information  provided  by  the  program  office.  We  clarified  the 
number  of  critical  manufacturing  processes  and  the  percentage  of 
statistical  process  control  where  information  existed  that  raised  concerns. 
We  used  a  standard  called  the  Process  Capability  Index,  a  process- 
performance  measurement  that  quantifies  how  closely  a  process  is 
running  to  its  specification  limits.  The  index  can  be  translated  into  an 
expected  product  defect  rate,  and  we  have  found  it  to  be  a  best  practice. 
We  also  used  data  provided  by  the  program  offices  on  their 
manufacturing  readiness  levels  (MRL)  for  process  capability  and  control, 
a  sub-thread  tracked  as  part  of  the  manufacturing  readiness  assessment 
process  recommended  by  DOD,  to  determine  production  maturity.  We 
assessed  programs  as  having  mature  manufacturing  processes  if  they 
reported  an  MRL  9  for  that  sub-thread — meaning,  that  manufacturing 
processes  are  stable,  adequately  controlled,  and  capable.  To  gain  further 
insights  into  production  maturity,  we  asked  program  officials  whether  the 
program  planned  to  demonstrate  critical  manufacturing  processes  on  a 
pilot  production  line  before  beginning  low-rate  production.  We  also  asked 
programs  on  what  date  they  planned  to  begin  system-level  developmental 
testing  of  a  fully  configured,  production-  representative  prototype  in  its 
intended  environment.  We  compared  this  date  to  the  production  start 
date.  We  did  not  assess  production  maturity  for  shipbuilding  programs. 

Although  the  knowledge  points  provide  excellent  indicators  of  potential 
risks,  by  themselves  they  do  not  cover  all  elements  of  risk  that  a  program 
encounters  during  development,  such  as  funding  instability.  Our  detailed 
reviews  on  individual  systems  normally  provide  a  more  comprehensive 
assessment  of  risk  elements. 
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We  conducted  this  performance  audit  from  June  2015  to  March  2016,  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 
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Table  7  contains  the  current  and  first  full  total  acquisition  cost  estimates 
(in  fiscal  year  2016  dollars)  for  each  program  or  element  in  the 
Department  of  Defense’s  (DOD)  2015  major  defense  acquisition  program 
portfolio.  For  each  program  we  show  the  percent  change  in  total 
acquisition  cost  from  the  first  full  estimate,  as  well  as  over  the  past  year 
and  5  years. 


Table  7:  Current  Cost  Estimates  and  First  Full  Estimates  for  DOD’s  2015  Portfolio  of  Major  Defense  Acquisition  Programs 


Program  name 

Current 

total 

acquisition 

cost 

First  full 
estimate 
total 
acquisition 
cost 

Change  in 
total 
acquisition 
cost  from 
first  full 
estimate 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past 
year 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  5 
years 
(percent) 

Advanced  Extremely  High  Frequency  (AEHF)  Satellite 

$14,795 

$6,835 

116.5% 

0.9% 

5.2% 

AGM-88E  Advanced  Anti-Radiation  Guided  Missile  (AGM- 
88E  AARGM) 

$2,274 

$1,718 

32.4% 

-0.2% 

14.5% 

AH-64E  Apache  New  Build  (AH-64E  New  Build) 

$2,319 

$2,542 

-8.8% 

0.4% 

-8.8% 

AH-64E  Apache  Remanufacture  (AH-64E  Remanufacture) 

$13,992 

$7,771 

80.1% 

-0.6% 

21.5% 

AIM-120  Advanced  Medium  Range  Air-to-Air  Missile 
(AMRAAM) 

$25,069 

$11,727 

113.8% 

0.8% 

-3.7% 

AIM-9X  Block  II  Air-to-Air  Missile  (AIM-9X  Block  II) 

$3,506 

$4,287 

-18.2% 

-4.5% 

-18.2% 

Air  and  Missile  Defense  Radar  (AMDR) 

$5,232 

$5,998 

-12.8% 

-0.5% 

-12.8% 

Airborne  and  Maritime/Fixed  Station  Joint  Tactical  Radio 
System  (AMF  JTRS) 

$3,590 

$8,748 

-59.0% 

2.0% 

-59.9% 

Airborne  Warning  and  Control  System  Block  40/45 

Upgrade  (AWACS  Blk  40/45  Upgrade) 

$2,837 

$2,996 

-5.3% 

-0.1% 

-5.3% 

B-2  Extremely  High  Frequency  SATCOM  and  Computer 
Increment  1  (B-2  EHF  Incl) 

$589 

$762 

-22.7% 

0.5% 

-12.6% 

B61  Mod  12  Life  Extension  Program  Tailkit  Assembly  (B61 
Mod  12  LEP  TKA) 

$1,377 

$1,403 

-1.8% 

-1 .5% 

-1 .8% 

C-130J  Hercules  Transport  Aircraft  (C-130J) 

$17,056 

$1,018 

1575.1% 

0.5% 

2.2% 

C-5  Reliability  Enhancement  and  Re-engineering  Program 
(C-5  RERP) 

$7,563 

$11,701 

-35.4% 

-0.4% 

-5.5% 

CH-47F  Improved  Cargo  Helicopter  (CH-47F) 

$16,032 

$3,454 

364.1% 

0.2% 

8.8% 

CH-53K  Heavy  Lift  Replacement  Helicopter  (CH-53K) 

$25,708 

$17,765 

44.7% 

0.2% 

7.8% 

Chemical  Demilitarization-Assembled  Chemical  Weapons 
Alternatives  (Chem  Demil-ACWA) 

$11,071 

$2,835 

290.5% 

0.4% 

28.0% 

Combat  Rescue  Helicopter  (CRH) 

$8,426 

$8,335 

1.1% 

1.1% 

1.1% 

Cooperative  Engagement  Capability  (CEC) 

$5,777 

$3,158 

82.9% 

2.2% 

4.8% 

DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 

$23,066 

$37,341 

-38.2% 

1 .2% 

6.9% 

Page  182 


GAO-1 6-329SP  Assessments  of  Major  Weapon  Programs 


Appendix  II:  Current  and  First  Full  Estimates 
for  DOD’s  2015  Portfolio  of  Major  Defense 
Acquisition  Programs 


Program  name 

Current 

total 

acquisition 

cost 

First  full 
estimate 
total 
acquisition 
cost 

Change  in 
total 
acquisition 
cost  from 
first  full 
estimate 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past 
year 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  5 
years 
(percent) 

DDG  51  Arleigh  Burke  Class  Guided  Missile  Destroyer 
(DDG51) 

$115,169 

$16,297 

606.7% 

2.9% 

12.1% 

E-2D  Advanced  Hawkeye  Aircraft  (E-2D  AHE) 

$21,538 

$15,827 

36.1% 

1.1% 

10.9% 

EA-18G  Growler  Aircraft  (EA-18G) 

$15,305 

$9,630 

58.9% 

11.1% 

21.2% 

Enhanced  Polar  System  (EPS) 

$1,425 

$1,431 

-0.4% 

-0.4% 

-0.4% 

Evolved  Expendable  Launch  Vehicle  (EELV) 

$60,497 

$18,642 

224.5% 

-2.7% 

224.5% 

Excalibur  Precision  155mm  Projectiles  (Excalibur) 

$1,962 

$5,125 

-61.7% 

0.3% 

-26.1% 

F-22  Increment  3.2B  Modernization  (F-22  Inc  3.2B  Mod) 

$1,565 

$1,608 

-2.6% 

-1 .3% 

-2.6% 

F-35  Joint  Strike  Fighter  (F-35) 

$339,997 

$229,285 

48.3% 

-1 .0% 

10.1% 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB- 
T) 

$4,308 

$3,421 

25.9% 

-10.6% 

0.7% 

Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78) 

$36,438 

$38,172 

-4.5% 

-0.9% 

-2.1% 

Global  Broadcast  Service  (GBS) 

$1,291 

$618 

109.0% 

2.3% 

6.0% 

Global  Positioning  System  III  (GPS  III) 

$4,918 

$4,229 

16.3% 

4.5% 

9.0% 

Ground/Air  Task  Oriented  Radar  (G/ATOR) 

$2,775 

$1,573 

76.5% 

0.0% 

76.5% 

Guided  Multiple  Launch  Rocket  System/Guided  Multiple 
Launch  Rocket  Sys  Alt  Warhead  (GMLRS/GMLRS  AW) 

$6,863 

$1,897 

261.8% 

-2.1% 

9.3% 

H-1  Upgrades  (4BW/4BN)  (H-1  Upgrades) 

$13,211 

$3,891 

239.5% 

-2.3% 

2.2% 

HC/MC-130  Recapitalization  Aircraft  (HC/MC-130  Recap) 

$14,424 

$8,974 

60.7% 

-0.7% 

60.7% 

Integrated  Air  and  Missile  Defense  (IAMD) 

$6,371 

$5,395 

18.1% 

0.8% 

18.1% 

Integrated  Defensive  Electronic  Countermeasures  (IDECM) 

$2,680 

$2,336 

14.7% 

0.5% 

10.7% 

IDECM  Block  4 

$923 

$745 

24.0% 

1 .5% 

22.5% 

IDECM  Blocks  2/3 

$1,757 

$1,591 

10.4% 

-0.1% 

5.3% 

Intercontinental  Ballistic  Missile  Fuze  Modernization  (ICBM 
Fuze  Mod) 

$1,882 

$1,870 

0.7% 

0.7% 

0.7% 

Joint  Air-to-Surface  Standoff  Missile  (JASSM) 

$7,346 

$2,487 

195.4% 

0.7% 

-6.4% 

Joint  Direct  Attack  Munition  (JDAM) 

$9,205 

$3,668 

151.0% 

10.4% 

32.5% 

Joint  Light  Tactical  Vehicle  (JLTV) 

$23,956 

$24,178 

-0.9% 

-0.3% 

-0.9% 

Joint  Precision  Approach  and  Landing  System  Increment 

1A  (JPALS  Inc  1  A) 

$1,590 

$1,086 

46.4% 

-1 .8% 

50.3% 

Joint  Standoff  Weapon  -  Baseline  Variant  and  Unitary 
Warhead  Variant  (JSOW) 

$4,389 

$8,527 

-48.5% 

-26.2% 

-24.4% 

JSOW  -  Baseline  Varient 

$2,215 

$3,065 

-27.7% 

-7.4% 

-7.8% 

JSOW  -  Unitary  Warhead  Varient 

$2,174 

$5,462 

-60.2% 

-38.9% 

-36.1% 

Joint  Tactical  Networks  (JTN) 

$2,316 

$1,052 

120.1% 

-0.3% 

6.6% 
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Program  name 

Current 

total 

acquisition 

cost 

First  full 
estimate 
total 
acquisition 
cost 

Change  in 
total 
acquisition 
cost  from 
first  full 
estimate 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past 
year 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  5 
years 
(percent) 

Handheld,  Manpack,  and  Small  Form  Fit  Radios  (HMS) 

$9,130 

$10,769 

-15.2% 

-13.0% 

75.2% 

KC-130J  Transport  Aircraft  (KC-130J) 

$9,739 

$10,176 

-4.3% 

-5.9% 

-4.3% 

KC-46  Tanker  Modernization  Program  (KC-46A) 

$43,532 

$47,021 

-7.4% 

-0.4% 

-7.4% 

LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6) 

$9,971 

$3,412 

192.2% 

-2.5% 

43.3% 

Littoral  Combat  Ship  -  Mission  Packages  (LCS  MP) 

$6,930 

$7,031 

-1.4% 

2.0% 

-1 .4% 

Littoral  Combat  Ship  (LCS) 

$20,954 

$2,408 

NA 

-3.0% 

NA 

LPD  17  San  Antonio  Class  Amphibious  Transport  Dock 
(LPD  17) 

$21,283 

$12,567 

69.4% 

6.2% 

6.4% 

M109A7  Family  of  Vehicles 

$7,201 

$7,176 

0.4% 

0.3% 

0.4% 

MH-60R  Multi-Mission  Helicopter  (MH-60R) 

$14,574 

$5,940 

145.4% 

5.3% 

-6.7% 

Mobile  User  Objective  System  (MUOS) 

$7,719 

$7,211 

7.0% 

0.2% 

3.8% 

MQ-1C  Gray  Eagle  Unmanned  Aircraft  System  (MQ-1C 
Gray  Eagle) 

$5,253 

$1,089 

382.2% 

7.5% 

-3.2% 

MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C  Triton) 

$12,815 

$13,783 

-7.0% 

-7.2% 

-9.7% 

MQ-8  (Fire  Scout) 

$2,918 

$2,806 

4.0% 

-5.9% 

8.5% 

MQ-9  Reaper  Unmanned  Aircraft  System  (MQ-9  Reaper) 

$12,154 

$2,829 

329.6% 

3.8% 

0.3% 

Multifunctional  Information  Distribution  System  (MIDS) 

$4,098 

$1,399 

192.9% 

2.7% 

28.0% 

Navy  Multiband  Terminal  (NMT) 

$2,132 

$2,490 

-14.4% 

6.9% 

-2.3% 

Next  Generation  Operational  Control  System  (GPS  OCX) 

$3,737 

$3,553 

5.2% 

5.4% 

5.2% 

P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  (P-8A) 

$33,269 

$33,297 

-0.1% 

-0.5% 

-5.6% 

Patriot  Advanced  Capability-3  Missile  Enhancement  (PAC- 
3  MSE) 

$6,354 

$7,818 

-18.7% 

-23.5% 

-24.0% 

Remote  Minehunting  System  (RMS) 

$1,574 

$1,547 

1.8% 

2.3% 

13.4% 

RQ-4A/B  Global  Hawk  Unmanned  Aircraft  System  (RQ- 
4A/B  Global  Hawk) 

$10,039 

$5,786 

73.5% 

-1 .3% 

-32.1% 

Ship  to  Shore  Connector  Amphibious  Craft  (SSC) 

$4,087 

$4,242 

-3.7% 

-0.5% 

-3.7% 

Small  Diameter  Bomb  Increment  II  (SDB  II) 

$3,970 

$5,045 

-21.3% 

0.5% 

-21.3% 

Space  Based  Infrared  System  High  (SBIRS  High) 

$19,135 

$4,932 

288.0% 

1.1% 

10.1% 

Space  Fence  Ground-Based  Radar  System  Increment  1 

$1,588 

$1,614 

-1.6% 

-1 .6% 

-1 .6% 

SSN  774  Virginia  Class  Submarine  (SSN  774) 

$91,144 

$64,857 

40.5% 

5.5% 

1 .8% 

Standard  Missile-6  (SM-6) 

$8,963 

$6,116 

46.6% 

-3.9% 

34.2% 

Tactical  Tomahawk  RGM-109E/UGM  109E  Missile 
(TACTOM) 

$6,255 

$2,270 

175.5% 

5.5% 

-16.1% 

Trident  II  (D-5)  Sea-Launched  Ballistic  Missile  UGM  133A 
(Trident  II  Missile) 

$57,803 

$55,504 

4.1% 

0.1% 

3.2% 
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Program  name 

Current 

total 

acquisition 

cost 

First  full 
estimate 
total 
acquisition 
cost 

Change  in 
total 
acquisition 
cost  from 
first  full 
estimate 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past 
year 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  5 
years 
(percent) 

UH-60M  Black  Hawk  Helicopter  (UH-60M  Black  Hawk) 

$26,273 

$13,917 

88.8% 

2.4% 

10.0% 

V-22  Osprey  Joint  Services  Advanced  Vertical  Lift  Aircraft 
(V- 22) 

$61,891 

$43,023 

43.9% 

0.2% 

1 .4% 

VH-92A  Presidential  Helicopter  Replacement  Program 

$4,817 

$4,790 

0.6% 

0.6% 

0.6% 

Warfighter  Information  Network-Tactical  Increment  2  (WIN- 
T  Inc  2) 

$10,433 

$3,979 

162.2% 

-17.8% 

102.0% 

Warfighter  Information  Network-Tactical  Increment  3  (WIN- 
T  Inc  3) 

$2,025 

$17,560 

-88.5% 

-45.2% 

-86.3% 

Wideband  Global  SATCOM  (WGS) 

$4,109 

$1,281 

220.8% 

-1.1% 

6.0% 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 


Notes:  Data  were  obtained  from  DOD's  Selected  Acquisition  Reports,  acquisition  program  baselines, 
and,  in  some  cases,  program  offices. 
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Appendix  III:  Changes  in  DOD’s  2015 
Portfolio  of  Major  Defense  Acquisition 
Programs  over  Five  Years  and  Since  First 
Full  Estimates 


Table  8  shows  the  change  in  research  and  development  cost, 
procurement  cost,  total  acquisition  cost,  and  average  delay  in  delivering 
initial  operational  capability  for  those  programs  in  Department  of 
Defense’s  (DOD)  2015  portfolio  over  the  last  5  years  and  since  their  first 
full  cost  and  schedule  estimates. 


Table  8:  Cost  and  Schedule  Changes  for  Programs  in  DOD’s  2015  Portfolio 

Fiscal  year  2016  dollars  (in  billions) 

5-year 

comparison 

(2010-2015) 

Since  first  full  estimate 
(Baseline  to  2015) 

Change  in  total  research  and  development 

$22.1 

$102.2 

cost 

8.3% 

54.7% 

Change  in  total  procurement  cost 

$70.8 

6.6% 

$364.9 

47.2% 

Change  in  total  other  acquisition  costs 

$2.8 

27.9% 

$1.6 

14.2% 

Change  in  total  acquisition  cost3 

$95.7 

7.1% 

$468.7 

48.3% 

Average  delay  in  delivering  initial  capabilities 

10.8  months 

11.1% 

29.5  months 

36.6% 

Source:  GAO  analysis  of  DOD  data.  |  GAO-1 6-329SP 

Notes:  Data  were  obtained  from  DOD's  Selected  Acquisition  Reports  and  acquisition  program 
baselines.  In  a  few  cases  data  were  obtained  directly  from  program  offices.  Some  numbers  may  not 
sum  due  to  rounding. 

aln  addition  to  research  and  development  and  procurement  costs,  total  acquisition  cost  includes 
acquisition-related  operation  and  maintenance  and  system-specific  military  construction  costs. 
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Practices 


GAO’s  prior  work  on  best  product-development  practices  found  that 
successful  programs  take  steps  to  gather  knowledge  that  confirms  that 
their  technologies  are  mature,  their  designs  stable,  and  their  production 
processes  are  in  control.  Successful  product  developers  ensure  a  high 
level  of  knowledge  is  achieved  at  key  junctures  in  development.  We 
characterize  these  junctures  as  knowledge  points.  The  Related  GAO 
Products  section  of  this  report  includes  references  to  the  body  of  work 
that  helped  us  identify  these  practices  and  apply  them  as  criteria  in 
weapon  system  reviews.  Table  9  summarizes  these  knowledge  points 
and  associated  key  practices. 


Table  9:  Best  Practices  for  Knowledge-Based  Acquisitions 


Knowledge  Point  1:  Technologies,  time,  funding,  and  other  resources  match  customer  needs.  Decision  to  invest  in  product 
development 

Demonstrate  technologies  to  a  high  readiness  level — Technology  Readiness  Level  7 — to  ensure  technologies  will  work  in  an 
operational  environment3 

Ensure  that  requirements  for  product  increment  are  informed  by  preliminary  design  review  using  systems  engineering  process 
(such  as  prototyping  of  preliminary  design) 

Establish  cost  and  schedule  estimates  for  product  on  the  basis  of  knowledge  from  preliminary  design  using  systems  engineering 
tools  (such  as  prototyping  of  preliminary  design) 

Constrain  development  phase  (5  to  6  years  or  less)  for  incremental  development 
Ensure  development  phase  fully  funded  (programmed  in  anticipation  of  milestone) 

Align  program  manager  tenure  to  complete  development  phase 

Contract  strategy  that  separates  system  integration  and  system  demonstration  activities 

Conduct  independent  cost  estimate 

Conduct  independent  program  assessment 

Conduct  major  milestone  decision  review  for  development  start 

Knowledge  Point  2:  Design  is  stable  and  performs  as  expected.  Decision  to  start  building  and  testing  production- 
representative  prototypes 

Complete  system  critical  design  review 

Complete  90  percent  of  engineering  design  drawing  packages 

Complete  subsystem  and  system  design  reviews 

Demonstrate  with  system-level  integrated  prototype  that  design  meets  requirements 
Complete  the  failure  modes  and  effects  analysis 
Identify  key  system  characteristics 
Identify  critical  manufacturing  processes 

Establish  reliability  targets  and  growth  plan  on  the  basis  of  demonstrated  reliability  rates  of  components  and  subsystems 
Conduct  independent  cost  estimate 
Conduct  independent  program  assessment 

Conduct  major  milestone  decision  review  to  enter  system  demonstration 
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Knowledge  Point  3:  Production  meets  cost,  schedule,  and  quality  targets.  Decision  to  produce  first  units  for  customer 

Demonstrate  manufacturing  processes 

Build  and  test  production-representative  prototypes  to  demonstrate  product  in  intended  environment 
Test  production-representative  prototypes  to  achieve  reliability  goal 
Collect  statistical  process  control  data 

Demonstrate  that  critical  processes  are  capable  and  in  statistical  control 

Conduct  independent  cost  estimate 

Conduct  independent  program  assessment 

Conduct  major  milestone  decision  review  to  begin  production 


Source:  GAO.  GAO-1 6-329P 

aDOD  considers  Technology  Readiness  Level  6,  demonstrations  in  a  relevant  environment,  to  be 
appropriate  for  programs  entering  system  development;  therefore,  we  have  analyzed  programs 
against  this  measure  as  well. 
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Appendix  V:  Technology  Readiness  Levels 


Technology 
readiness  level 

Description 

Hardware/software 

Demonstration 

environment 

1. 

Basic  principles 
observed  and 
reported 

Lowest  level  of  technology  readiness. 

Scientific  research  begins  to  be  translated 
into  applied  research  and  development. 
Examples  might  include  paper  studies  of  a 
technology’s  basic  properties 

None  (paper  studies  and 
analysis) 

None 

2. 

Technology  concept 
and/or  application 
formulated 

Invention  begins.  Once  basic  principles  are 
observed,  practical  applications  can  be 
invented.  The  application  is  speculative  and 
there  is  no  proof  or  detailed  analysis  to 
support  the  assumption.  Examples  are  still 
limited  to  paper  studies. 

None  (paper  studies  and 
analysis) 

None 

3. 

Analytical  and 
experimental  critical 
function  and/or 
characteristic  proof  of 
concept 

Active  research  and  development  is  initiated. 
This  includes  analytical  studies  and 
laboratory  studies  to  physically  validate 
analytical  predictions  of  separate  elements  of 
the  technology.  Examples  include 
components  that  are  not  yet  integrated  or 
representative. 

Analytical  studies  and 
demonstration  of  non-scale 
individual  components  (pieces 
of  subsystem) 

Lab 

4. 

Component  and/or 
breadboard  validation 
in  laboratory 
environment 

Basic  technological  components  are 
integrated  to  establish  that  the  pieces  will 
work  together.  This  is  relatively  “low  fidelity” 
compared  to  the  eventual  system.  Examples 
include  integration  of  “ad  hoc”  hardware  in  a 
laboratory. 

Low-fidelity  breadboard. 
Integration  of  non-scale 
components  to  show  pieces  will 
work  together.  Not  fully 
functional  or  form  or  fit  but 
representative  of  technically 
feasible  approach  suitable  for 
flight  articles. 

Lab 

5. 

Component  and/or 
breadboard  validation 
in  relevant 
environment 

Fidelity  of  breadboard  technology  increases 
significantly.  The  basic  technological 
components  are  integrated  with  reasonably 
realistic  supporting  elements  so  that  the 
technology  can  be  tested  in  a  simulated 
environment.  Examples  include  “high  fidelity” 
laboratory  integration  of  components. 

High-fidelity  breadboard. 
Functionally  equivalent  but  not 
necessarily  form  and/or  fit  (size 
weight,  materials,  etc.).  Should 
be  approaching  appropriate 
scale.  May  include  integration 
of  several  components  with 
reasonably  realistic  support 
elements/subsystems  to 
demonstrate  functionality. 

Lab  demonstrating 
functionality  but  not 
form  and  fit.  May 
include  flight 
demonstrating 
breadboard  in  surrogate 
aircraft.  Technology 
ready  for  detailed 
design  studies. 

6. 

Syste  m/su  bsyste  m 
model  or  prototype 
demonstration  in  a 
relevant  environment 

Representative  model  or  prototype  system, 
which  is  well  beyond  the  breadboard  tested 
for  TRL  5,  is  tested  in  a  relevant  environment. 
Represents  a  major  step  up  in  a  technology’s 
demonstrated  readiness.  Examples  include 
testing  a  prototype  in  a  high  fidelity  laboratory 
environment  or  in  simulated  realistic 
environment. 

Prototype.  Should  be  very  close 
to  form,  fit  and  function. 

Probably  includes  the 
integration  of  many  new 
components  and  realistic 
supporting 

elements/subsystems  if  needed 
to  demonstrate  full  functionality 
of  the  subsystem. 

High-fidelity  lab 
demonstration  or 
limited/restricted  flight 
demonstration  for  a 
relevant  environment. 
Integration  of 
technology  is  well 
defined. 

7. 

System  prototype 
demonstration  in  an 
operational 
environment 

Prototype  near  or  at  planned  operational 
system.  Represents  a  major  step  up  from 

TRL  6,  requiring  the  demonstration  of  an 
actual  system  prototype  in  a  realistic 
environment,  such  as  in  an  aircraft,  vehicle  or 

Prototype.  Should  be  form,  fit 
and  function  integrated  with 
other  key  supporting 
elements/subsystems  to 
demonstrate  full  functionality  of 

Flight  demonstration  in 
representative  realistic 
environment  such  as 
flying  test  bed  or 
demonstrator  aircraft. 
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Technology 
readiness  level 

Description 

Hardware/software 

Demonstration 

environment 

space.  Examples  include  testing  the 
prototype  in  a  test  bed  aircraft. 

subsystem. 

Technology  is  well 
substantiated  with  test 
data. 

8.  Actual  system 

completed  and  “flight 
qualified”  through  test 
and  demonstration 

Technology  has  been  proven  to  work  in  its 
final  form  and  under  expected  conditions.  In 
almost  all  cases,  this  TRL  represents  the  end 
of  true  system  development.  Examples 
include  developmental  test  and  evaluation  of 
the  system  in  its  intended  weapon  system  to 
determine  if  it  meets  design  specifications. 

Flight-qualified  hardware 

Developmental  Test  and 
Evaluation  in  the  actual 
system  application. 

9.  Actual  system  “flight 
proven”  through 
successful  mission 
operations 

Actual  application  of  the  technology  in  its  final 
form  and  under  mission  conditions,  such  as 
those  encountered  in  operational  test  and 
evaluation.  In  almost  all  cases,  this  is  the  end 
of  the  last  “bug  fixing”  aspects  of  true  system 
development.  Examples  include  using  the 
system  under  operational  mission  conditions. 

Actual  system  in  final  form 

Operational  Test  and 
Evaluation  in 
operational  mission 
conditions. 

Source:  GAO  and  its  analysis  of  National  Aeronautics  and  Space  Administration  data.  GAO-1 6-329SP 
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Appendix  VI:  Major  Defense  Acquisition 
Programs’  Individual  Subcontracting  Reports 
in  the  Electronic  Subcontracting  Reporting 
System 

Table  10  shows  the  number  of  prime  contractors  for  the  programs  we 
assessed  where  an  individual  subcontracting  report  is  reported  as 
accepted  during  2015  in  the  Electronic  Subcontracting  Reporting  System 
(eSRS).  We  reviewed  this  information  for  79  major  defense  acquisition 
programs  included  in  our  assessment  that  reported  prime  contract 
information  in  their  December  2014  Selected  Acquisition  Report  (SAR) 
submissions.  The  government  uses  individual  subcontracting  reports  from 
eSRS  as  one  method  of  monitoring  small  business  participation,  as  this 
tool  includes  information  on  contractors’  performance  against  small 
business  subcontracting  goals.  There  are  multiple  reasons  why  a  prime 
contractor  may  not  have  an  accepted  subcontracting  report  in  eSRS.  For 
example,  some  contractors  may  have  pending  or  rejected  reports  within 
the  system  as  all  reports  are  reviewed  prior  to  acceptance.  Not  all  prime 
contracts  for  major  defense  acquisition  programs  are  required  to  submit 
individual  subcontracting  reports.  Instead,  some  contractors  report  small 
business  participation  at  a  corporate  level  as  opposed  to  the  program 
level  and  this  data  is  not  captured  in  the  individual  subcontracting 
reports.1  In  addition,  although  a  prime  contractor  may  be  required  to 
submit  a  report,  it  may  not  yet  have  done  so  for  the  period  we  reviewed.2 


of  November  2015,  12  major  defense  companies  were  participating  in  the  Test 
Program  for  Negotiation  of  Comprehensive  Small  Business  Subcontracting  Plans  created 
by  the  National  Defense  Authorization  Act  for  Fiscal  Years  1 990  and  1991,  Pub.  L.  No. 
101-189,  §  834  (1989).  One  additional  company  ceased  participation  in  2007,  although 
active  contracts  awarded  before  that  time  are  still  reported  through  the  test  program. 
These  major  defense  companies  have  each  established  a  comprehensive  subcontracting 
plan  on  a  corporate,  division  or  plant-wide  basis  under  which  a  single  summary 
subcontract  report  is  submitted  semi-annually  for  any  covered  DOD  contracts.  The  test 
program  has  been  extended  by  Congress  several  times  with  the  current  three  year 
extension  made  by  Pub.  L.  No.  113-291,  §  821  (2014)  to  end  on  December  31, 2017. 
Participation  in  the  test  program  is  on  a  voluntary  basis  such  that  these  major  defense 
companies  may  have  contracts  where  they  are  reporting  on  an  individual  basis  as  well  as 
contracts  where  they  are  reporting  on  a  comprehensive  basis. 

2For  further  information  on  the  limitations  of  eSRS  and  other  contract  reporting  systems, 
see  GAO,  Federal  Subcontracting:  Linking  Small  Business  Subcontractors  to  Prime 
Contracts  Is  Not  Feasible  Using  Current  Systems,  GAO-15-1 16  (Washington,  D.C.:  Dec. 
11,2014). 
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Table  10:  Major  Defense  Acquisition  Programs’  Individual  Subcontracting  Reports  in  the  Electronic  Subcontracting  Reporting 
System 

Program  name 

Number  of  contracts  listed 
in  the  December  2014  SAR 

Contracts  with  an  accepted 
individual  subcontracting 
report  (as  of  December  2015) 

Advanced  Extremely  High  Frequency  (AEHF)  Satellite 

2 

0 

AGM-88E  Advanced  Anti-Radiation  Guided  Missile  (AGM-88E 
AARGM) 

2 

2 

AH-64E  Apache  New  Build  (AH-64E  New  Build) 

1 

1 

AH-64E  Apache  Remanufacture  (AH-64E  Remanufacture) 

5 

2 

AIM-9X  Block  II  Air-to-Air  Missile  (AIM-9X  Block  II) 

4 

0 

Air  and  Missile  Defense  Radar  (AMDR) 

1 

0 

Airborne  and  Maritime/Fixed  Station  Joint  Tactical  Radio  System 
(AMF  JTRS) 

0 

0 

AIM-120  Advanced  Medium  Range  Air-to-Air  Missile  (AMRAAM) 

6 

0 

Airborne  Warning  and  Control  System  Block  40/45  Upgrade  (AWACS 
Blk  40/45  Upgrade) 

1 

0 

B-2  Extremely  High  Frequency  SATCOM  and  Computer  Increment  1 
(B-2  EHF  Incl ) 

1 

1 

B61  Mod  12  Life  Extension  Program  Tailkit  Assembly  (B61  Mod  12 
LEP  TKA) 

1 

1 

C-130J  Hercules  Transport  Aircraft  (C-130J) 

4 

0 

C-5  Reliability  Enhancement  and  Re-engineering  Program  (C-5 

RERP) 

1 

0 

Cooperative  Engagement  Capability  (CEC) 

4 

0 

CH-47F  Improved  Cargo  Helicopter  (CH-47F) 

1 

1 

CH-53K  Heavy  Lift  Replacement  Helicopter  (CH-53K) 

2 

0 

Chemical  Demilitarization-Assembled  Chemical  Weapons 

Alternatives  (Chem  Demil-ACWA) 

2 

2 

Combat  Rescue  Helicopter  (CRH) 

1 

0 

Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78) 

4 

3 

DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 

3 

3 

DDG  51  Arleigh  Burke  Class  Guided  Missile  Destroyer  (DDG  51) 

6 

5 

E-2D  Advanced  Hawkeye  Aircraft  (E-2D  AHE) 

5 

2 

EA-18G  Growler  Aircraft  (EA-18G) 

6 

4 

Evolved  Expendable  Launch  Vehicle  (EELV) 

4 

3 

Enhanced  Polar  System  (EPS) 

1 

1 

Excalibur  Precision  155mm  Projectiles  (Excalibur) 

2 

0 

F-22  Increment  3.2B  Modernization  (F-22  Inc  3.2B  Mod) 

1 

0 

F-35  Joint  Strike  Fighter  (F-35) 

6 

0 
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Program  name 

Number  of  contracts  listed 
in  the  December  2014  SAR 

Contracts  with  an  accepted 
individual  subcontracting 
report  (as  of  December  2015) 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T) 

2 

0 

MQ-8  (Fire  Scout) 

1 

1 

Ground/Air  Task  Oriented  Radar  (G/ATOR) 

1 

0 

Global  Broadcast  Service  (GBS) 

1 

0 

Guided  Multiple  Launch  Rocket  System/Guided  Multiple  Launch 
Rocket  Sys  Alt  Warhead  (GMLRS/GMLRS  AW) 

4 

0 

Global  Positioning  System  III  (GPS  III) 

2 

1 

Next  Generation  Operational  Control  System  (GPS  OCX) 

1 

0 

H-1  Upgrades  (4BW/4BN)  (H-1  Upgrades) 

5 

3 

HC/MC-130  Recapitalization  Aircraft  (HC/MC-130  Recap) 

3 

0 

Integrated  Air  and  Missile  Defense  (IAMD) 

2 

1 

Intercontinental  Ballistic  Missile  Fuze  Modernization  (ICBM  Fuze 

Mod) 

1 

0 

Integrated  Defensive  Electronic  Countermeasures  (IDECM)  Block  4 

3 

1 

Joint  Air-to-Surface  Standoff  Missile  (JASSM) 

3 

0 

Joint  Direct  Attack  Munition  (JDAM) 

2 

2 

Joint  Light  Tactical  Vehicle  (JLTV) 

3 

2 

Joint  Precision  Approach  and  Landing  System  Increment  1A  (JPALS 
IncIA) 

1 

0 

Joint  Standoff  Weapon  -  Baseline  and  Unitary  Warhead  Varient 
(JSOW) 

0 

0 

Joint  Tactical  Networks  (JTN) 

0 

0 

Joint  Tactical  Radio  System  Handheld,  Manpack,  and  Small  Form  Fit 
Radios  (JTRS  HMS) 

1 

0 

KC-130J  Transport  Aircraft  (KC-130J) 

4 

0 

KC-46  Tanker  Modernization  Program  (KC-46A) 

3 

3 

Littoral  Combat  Ship  (LCS) 

2 

1 

Littoral  Combat  Ship  -  Mission  Packages  (LCS  MP) 

1 

0 

LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6) 

1 

1 

LPD  17  San  Antonio  Class  Amphibious  Transport  Dock  (LPD  17) 

1 

1 

M109A7  Family  of  Vehicles 

2 

2 

MH-60R  Multi-Mission  Helicopter  (MH-60R) 

4 

0 

Multifunctional  Information  Distribution  System  (MIDS) 

2 

2 

MG-1C  Gray  Eagle  Unmanned  Aircraft  System  (MG-1C  Gray  Eagle) 

5 

4 

MG-4C  Triton  Unmanned  Aircraft  System  (MO-4C  Triton) 

1 

1 

MG-9  Reaper  Unmanned  Aircraft  System  (MG-9  Reaper) 

3 

0 

Mobile  User  Objective  System  (MUOS) 

1 

0 
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System 

Program  name 

Number  of  contracts  listed 
in  the  December  2014  SAR 

Contracts  with  an  accepted 
individual  subcontracting 
report  (as  of  December  2015) 

Navy  Multiband  Terminal  (NMT) 

1 

0 

P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  (P-8A) 

3 

2 

Patriot  Advanced  Capability-3  Missile  Enhancement  (PAC-3  MSE) 

3 

0 

Remote  Minehunting  System  (RMS) 

2 

0 

RQ-4A/B  Global  Hawk  Unmanned  Aircraft  System  (RQ-4A/B  Global 
Hawk) 

0 

0 

Space  Based  Infrared  System  High  (SBIRS  High) 

4 

0 

Small  Diameter  Bomb  Increment  II  (SDB  II) 

1 

0 

Standard  Missile-6  (SM-6) 

3 

0 

Space  Fence  Ground-Based  Radar  System  Increment  1 

1 

0 

Ship  to  Shore  Connector  Amphibious  Craft  (SSC) 

1 

1 

SSN  774  Virginia  Class  Submarine  (SSN  774) 

2 

2 

Tactical  Tomahawk  RGM-109E/UGM  109E  Missile  (TACTOM) 

2 

0 

Trident  II  (D-5)  Sea-Launched  Ballistic  Missile  UGM  133A  (Trident  II 
Missile) 

8 

2 

UH-60M  Black  Hawk  Helicopter  (UH-60M  Black  Hawk) 

1 

0 

V-22  Osprey  Joint  Services  Advanced  Vertical  Lift  Aircraft  (V-22) 

5 

4 

VH-92A  Presidential  Helicopter  Replacement  Program 

1 

0 

Wideband  Global  SATCOM  (WGS) 

1 

1 

Warfighter  Information  Network-Tactical  Increment  2  (WIN-T  Inc  2) 

1 

0 

Warfighter  Information  Network-Tactical  Increment  3  (WIN-T  Inc  3) 

1 

0 

Totals 

184 

69 

Source:  GAO  analysis  of  data  from  DOD  and  eSRS.  |  GAO-1 6-329SP 
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Appendix  VII:  Comments  from  the 
Department  of  Defense 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3000 


ACQUISITION.  MAR  9  2016 

TECHNOLOGY 
AND  LOGISTICS 

Mr.  Michael  J.  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  GStNW 
Washington,  DC  20548 

Dear  Mr.  Sullivan, 

This  is  the  Department  of  Defense  response  to  the  GAO  Draft  Report,  GAO-16-329SP. 
"Defense  Acquisitions:  Assessments  of  Selected  Weapon  Programs’'  dated  March  2016  (GAO 
Code  121276). 

The  Department  is  pleased  that  this  year's  Draft  Report,  the  14lh  annual  assessment  on  the 
performance  of  DoD’s  major  acquisition  programs,  identifies  many  ways  in  which  the 
Department  continues  to  drive  down  the  cost  of  the  acquisition  portfolio.  GAO  results  appear  to 
validate  our  focus  on  continuous  improvement  on  the  cost,  schedule,  and  performance  measures 
of  our  programs  through  employment  of  the  Better  Buying  Power  initiatives.  The  enclosed  chart 
illustrates  the  five-year  moving  average  of  the  annual  growth  of  contracted  costs  over  the  past 
thirty  years,  note  that  it  shows  a  steep  decline  in  contract  cost  growth  following  the 
implementation  of  Better  Buying  Power. 

The  Department  appreciates  the  opportunity  to  comment  on  the  Draft  Report.  My  point 
of  contact  for  this  effort  is  LCDR  Joseph  Mitzen,  703-697-8020. 

Sincerely, 

6C 

Nancy  L.  Spruill 
Director 

Acquisition  Resources  &  Analysis 

Enclosure: 

As  stated 


Page  195 


GAO-1 6-329SP 


Appendix  VIII:  GAO  Contact  and  Staff 
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GAO  Contact 


Michael  J.  Sullivan,  (202)  512-4841  orsullivanm@gao.gov 


Staff 

Acknowledgments 


Principal  contributors  to  this  report  were  J.  Kristopher  Keener,  Assistant 
Director;  Enyinnaya  David  Aja;  Desiree  E.  Cunningham;  Matthew  T. 
Drerup;  Danny  Owens;  and  Wendy  Smythe.  Other  key  contributors 
included,  Cheryl  Andrew,  Naba  Barkakati,  Emily  Bond,  Edwin  B.  Booth, 
Jennifer  A.  Dougherty,  Rich  Horiuchi,  Julia  Kennon,  Jill  N.  Lacey,  Kate 
Lenane,  Jean  L.  McSween,  Travis  J.  Masters,  LaTonya  Miller,  Diana  L. 
Moldafsky,  Scott  M.  Purdy,  Beth  Reed  Fritts,  Ronald  E.  Schwenn,  Bill 
Shear,  Charlie  Shivers,  Roxanna  T.  Sun,  Jay  Tallon,  Lindsay  C.  Taylor, 
Bruce  H.  Thomas,  Oziel  A.  Trevino,  and  Alyssa  Weir.  The  following  were 
responsible  for  individual  programs: 


Current  programs 

Program  name 

Primary  staff 

AIM-9X  Block  II  Air-to-Air  Missile 

Jennifer  A.  Dougherty,  Stephen  V.  Marchesani,  Cale  Jones 

Air  and  Missile  Defense  Radar  (AMDR) 

Sean  D.  Merrill,  Laura  M.  Jezewski,  Nathan  Foster 

Airborne  &  Maritime/Fixed  Station  (AMF) 

Lindsay  C.  Taylor,  Scott  M.  Purdy 

Amphibious  Combat  Vehicle  (ACV) 

Jenny  Shinn,  Zachary  Sivo,  Betsy  Gregory-Hosler 

Armored  Multi-Purpose  Vehicle  (AMPV) 

Andrea  M.  Bivens,  Marcus  C.  Ferguson 

B-2  Defensive  Management  System  Modernization  (B-2  DMS-M) 

Matthew  B.  Lea,  Don  M.  Springman 

CH-53K  Heavy  Lift  Replacement  Helicopter 

Robert  K.  Miller,  Brian  T.  Smith 

Combat  Rescue  Helicopter  (CRH) 

Sean  C.  Seales,  J.  Andrew  Walker 

Common  Infrared  Countermeasure  (CIRCM) 

Danny  Owens,  Wendy  Smythe 

CVN  78  Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78) 

Burns  C.  Eckert,  Abby  C.  Volk,  Kelsey  Hawley 

DDG  51  Flight  III  Destroyer  (DDG  51  Flight  III) 

Laura  M.  Jezewski,  Sean  D.  Merrill,  Nathan  Foster 

DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 

Angie  Nichols-Friedman,  Patrick  Breiding,  Garrett  Riba 

Enhanced  Polar  System  (EPS) 

Bradley  L.  Terry,  Maricela  Cherveny 

Evolved  Expendable  Launch  Vehicle  (EELV) 

Andrew  Redd,  Erin  E.  Cohen,  Desiree  E.  Cunningham 

F-22  Increment  3.2B  Modernization  (F-22  Inc  3.2B  Mod) 

Julie  C.  Hadley,  Robert  P.  Bullock 

F-35  Joint  Strike  Fighter  Aircraft  (F-35  JSF) 

Megan  Sester,  Jillena  S.  Roberts,  Samuel  Woo 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T) 

Alexandra  Dew  Silva,  Jessica  E.  Karnis,  Justin  Jaynes 

Global  Positioning  System  III  (GPS  III) 

Laura  T.  Holliday,  Raj  Chitikila 

Ground/Air  Task  Oriented  Radar  (G/ATOR) 

Joe  Hunter,  Bonita  J.  P.  Oden 

Handheld,  Manpack,  and  Small  Form  Fit  Radios  (HMS) 

Scott  M.  Purdy,  Jessica  E.  Karnis 

Integrated  Air  and  Missile  Defense  (IAMD) 

Carol  T.  Mebane,  Meredith  A.  Kimmett 

Joint  Air-to-Ground  Missile  (JAGM) 

Daniel  Singleton,  Wendy  Smythe 

Joint  Light  Tactical  Vehicle  (JLTV) 

Marcus  C.  Ferguson,  Andrea  M.  Bivens 
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Program  name 

Primary  staff 

KC-46A  Tanker  Modernization  Program  (KC-46A) 

Katheryn  S.  Hubbell,  Nathaniel  Vaught  Jeffrey  L.  Hartnett 

LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6) 

Abby  C.  Volk,  Teague  A.  Lyons,  Matthew  Jacobs 

Littoral  Combat  Ship  (LCS) 

Jacob  Beier,  C.  James  Madar 

Littoral  Combat  Ship  -  Mission  Modules  (LCS  Packages) 

Laurier  R.  Fish,  Jacob  Beier 

M109A7  Family  of  Vehicles  (M109A7  FOV) 

William  C.  Allbritton,  Richard  Cederholm 

Military  Global  Positioning  System  User  Equipment  Increment  1  (MGUE) 

Raj  Chitikila,  Andrew  Redd 

MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C) 

Don  M.  Springman,  Tom  Twambly 

MQ-8  Fire  Scout  Unmanned  Aircraft  System  (MQ-8  Fire  Scout) 

James  Kim,  Justin  Jaynes,  Jeffrey  Hamer 

Next  Generation  Jammer  Increment  1  (NGJ  Increment  1) 

Teakoe  S.  Coleman,  Laura  T.  Holliday 

Next  Generation  Operational  Control  System  (OCX) 

Claire  A.  Buck,  Erin  R.  Cohen,  Maricela  Cherveny 

Offensive  Anti-Surface  Warfare  Increment  1  (OASuW  Inc  1) 

Tom  Twambly,  Laurier  R.  Fish,  Ji  Byun 

Patriot  Advanced  Capability-3  Missile  Segment  Enhancement  (PAC-3 
MSE) 

Meredith  A.  Kimmett,  Carol  T.  Mebane 

Ship  to  Shore  Connector  Amphibious  Craft  (SSC) 

Teague  A.  Lyons,  Beth  Reed  Fritts,  Molly  Callaghan 

Small  Diameter  Bomb  Increment  II  (SDB  II) 

John  W.  Crawford,  Carrie  W.  Rogers 

Space  Fence  Ground  Based  Radar  System  Increment  1  (Space  Fence 

Inc  1) 

Laura  D.  Hook,  Suzanne  Sterling 

Three-Dimensional  Expeditionary  Long-Range  Radar  (3DELRR) 

Claire  Li,  Joe  Hunter 

VH-92A  Presidential  Helicopter  (VH-92A) 

Bonita  J.  P.  Oden,  Robert  K.  Miller 

Warfighter  Information  Network-Tactical  Increment  2  (WIN-T  Inc  2) 

Andrea  Yohe,  Ryan  Stott 

Warfighter  Information  Network-Tactical  Increment  3  (WIN-T  Inc  3) 

Ryan  Stott,  Andrea  Yohe 

Future  Programs 

Advanced  Pilot  Trainer  (APT) 

Marvin  E.  Bonner,  Sean  C.  Seales 

Amphibious  Ship  Replacement  (LX(R)) 

Holly  Williams,  Jenny  Shinn,  Zachary  Sivo 

F-15  Eagle  Passive  Active  Warning  Survivability  System 
(F-15  EPAWSS) 

Wendell  K.  Hudson,  LeAnna  M.  Parkey 

Improved  Turbine  Engine  Program  (ITEP) 

Wendy  Smythe,  Daniel  Singleton 

Indirect  Fire  Protection  Capability  Increment  2  -  Intercept,  Block  1  (IFPC 
Inc  2,  Block  1) 

Helena  Johnson,  James  P.  Haynes 

Joint  Surveillance  Target  Attack  Radar  System  Recapitalization  (JSTARS 
Recap) 

J.  Andrew  Walker,  Sameena  Ismailjee,  Alexandra  Stone 

Ohio-class  Replacement  (Ohio  Replacement) 

C.  James  Madar,  Christopher  J.  Yun 

P8-A  Increment  3  Upgrade  Program  (P8-A  Inc  3) 

Heather  B.  Miller  Jocelyn  C.  Yin 

Presidential  Aircraft  Recapitalization  (PAR) 

LeAnna  M.  Parkey,  Wendell  K.  Hudson 

Unmanned  Carrier  Launched  Airborne  Surveillance  and  Strike  (UCLASS) 
System 

Robert  P.  Bullock,  Julie  C.  Hadley 

Weather  Satellite  Follow-On  (WSF) 

Maricela  Cherveny,  Brenna  E.  Guarneros 
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Navy  Shipbuilding:  Significant  Investments  in  the  Littoral  Combat  Ship 
Continue  Amid  Substantial  Unknowns  about  Capabilities,  Use,  and 
Cost.  GAO-13-530.  Washington,  D.C.:  July  22,  2013. 

Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs.  GAO-1 3-294SP.  Washington,  D.C.:  March  28,  2013. 

High-Risk  Series:  An  Update.  GAO-15-290.  Washington,  D.C.:  February 
2015. 

Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs.  GAO-1 2-400SP.  Washington,  D.C.:  March  29,  2012. 

Arleigh  Burke  Destroyers:  Additional  Analysis  and  Oversight  Required  to 
Support  the  Navy’s  Future  Surface  Combatant  Plans.  GAO-12-1 13. 
Washington,  D.C.:  January  24,  2012. 

Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs.  GAO-1 1-233SP.  Washington,  D.C.:  March  29,  2011. 

Best  Practices:  DOD  Can  Achieve  Better  Outcomes  by  Standardizing  the 
Way  Manufacturing  Risks  Are  Managed.  GAO-10-439.  Washington,  D.C.: 
April  22,  2010. 

V-22  Osprey  Aircraft:  Assessments  Needed  to  Address  Operational  and 
Cost  Concerns  to  Define  Future  Investments.  GAO-09-692T.  Washington, 
D.C.:  June  23,  2009. 

Best  Practices:  High  Levels  of  Knowledge  at  Key  Points  Differentiate 
Commercial  Shipbuilding  from  Navy  Shipbuilding.  GAO-09-322. 
Washington,  D.C.:  May  13,  2009. 

GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for 
Developing  and  Managing  Capital  Program  Costs.  GAO-09-3SP. 
Washington,  D.C.:  March  2,  2009. 

Defense  Acquisitions:  A  Knowledge-Based  Funding  Approach  Could 
Improve  Major  Weapon  System  Program  Outcomes.  GAO-08-619. 
Washington,  D.C.:  July  2,  2008. 

Defense  Acquisitions:  Realistic  Business  Cases  Needed  to  Execute  Navy 
Shipbuilding  Programs.  GAO-07-943T.  Washington,  D.C.  July  24,  2007. 


Page  199 


GAO-1 6-329SP 


Related  GAO  Products 


Defense  Acquisitions:  Stronger  Management  Practices  are  Needed  to 
Improve  DOD’s  Software-Intensive  Weapon  Acquisitions.  GAO-04-393. 
Washington,  D.C.:  March  1, 2004. 

Best  Practices:  Using  a  Knowledge-Based  Approach  to  Improve  Weapon 
Acquisition.  GAO-04-386SP.  Washington,  D.C.:  January  2004. 

Multiyear  Procurement  Authority  for  the  Virginia  Class  Submarine 
Program.  GAO-03-895R.  Washington,  D.C.:  June  23,  2003. 

Best  Practices:  Capturing  Design  and  Manufacturing  Knowledge  Early 
Improves  Acquisition  Outcomes.  GAO-02-701.  Washington,  D.C.:  July  15, 
2002. 

Best  Practices:  Better  Matching  of  Needs  and  Resources  Will  Lead  to 
Better  Weapon  System  Outcomes.  GAO-01 -288.  Washington,  D.C.: 

March  8,  2001. 

Best  Practices:  Better  Management  of  Technology  Development  Can 
Improve  Weapon  System  Outcomes.  GAO/NSIAD-99-162.  Washington, 
D.C.:  July  30,  1999. 


(121276) 


Page  200 


GAO-1 6-329SP 


GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s 
website,  http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates. 

Listen  to  our  Podcasts  and  read  The  Watchblog. 

Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 
7125,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 

W 

Please  Print  on  Recycled  Paper. 

